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Foreword 


The halfway mark in the twentieth century was the beginning of the 
halcyon period of game theory. With the pioneering work of von 
Neumann and Morgenstern, The Theory of Games and Economic Behaviour, 
this new mathematical tool became the central focus of the research 
efforts of the major military analysis groups. Under the leadership of the 
Mathematics Department of the Rand Corporation, the theory was 
developed, expanded, and applied to military problems. 

Midcentury was also the time of inception of the guided interceptor 
missile and concentration on the general problem of pursuit and evasion. 
Is the best way to reach a moving target collision course or constant 
bearing navigation, direct or anticipatory pursuit? What is the best 
strategy for evasion? How can two airliners best cooperate to avert 
collision? 

From the forming and solving of military pursuit games in the early 
1950’s, the present theory grew. Reflecting on the general problem of 
pursuit, Dr. Isaacs realized that no one guidance scheme could be optimal 
against all types of evasion, for the evader can deliberately maneuver to 
confound the pursuer’s predictions. Optimal pursuit and evasion, then, 
must be considered collaterally and with parity; neither could have 
meaning without the other. Game theory was an essential element of the 
problem. 

If either craft is controlled in flight, by either a human or automatic 
pilot, there must be certain quantities continually under his volition. 
Such quantities are unpredictable to the opponent; a guidance scheme 
based on them is not only futile but against a clever foe may be harmful. 

There followed the vital distinction between state and control variables, 


vii 


ad a 











viii FOREWORD 

which was a key step to a general formulation. At once the nature of a 
strategy was clear: one makes the control variables functions of the state 
variables. Not only is such an immediate generalization of the strategies 
of discrete games, but it is also exactly a guidance scheme. 

It soon became evident that the resulting formulation was general 
enough for a wider range of material and Isaacs sought diverse conflict 
problems and adapted them to his new structure. It was at this time, 
roughly 1951, that the author coined the name differential games. 

From the outset Isaacs emphasizes the importance of obtaining answers. 
In Differential Games, A Mathematical Theory with Applications to 
Warfare and Pursuit, Control and Optimization, not only does he develop 
the theory of differential games, but his work is enhanced by a magnificent 
range of practical military and other applications with their solutions. 
This work is the culmination of fifteen years of reseach by the author 
and is likely to stand for many years as the definitive work on the subject. 


FRANK E. BOTHWELL 
Chief Scientist 

Center for Naval Analyses 
The Franklin Institute 


Washington, D.C. 
October, 1964 


Preface 


Although combat problems were its original motive, this book has 
turned out to be far from a manual of military techniques. Rather the 
result is a mathematical entity which fuses game theory, the calculus of 
variations, and control theory, and, because of its subsuming character, 
often transcends all three. 

As ideas burgeoned, they dictated their own course. Combat problems 
treated as genuine two-player games can be extremely difficult. (This 
matter is discussed at length in Chapter 11.) Their resolution demanded 
first a theory and it, one may logically say, is this book’s real contribution. 

But it grew from solving problems. Each seemed accompanied by 
strange, new phenomena and, as each conception was mastered, still 
newer ones proliferated. The unanticipated was and still is a fascinating 
aspect of differential games. Baffling novelties never seem to cease 
appearing, and it is therefore yet hard to say how complete the theory is. 

To the reader seeking an introduction or a superficial acquaintance with 
the subject, I suggest the following program: 


Chapter 1 depicts the nature and scope of differential games. It presents 
typical problems, but says nothing about the mathematics or techniques 
needed for their-solution. 

The latter can be sensed from Chapter 3, which is devoted to discrete 
models, some of them being quantized versions of problems later treated 
continuously. Because these discrete problems can be solved step by step, 
the reader can glimpse the concepts here without most of the formal 
mathematical tools. 

Chapter 2 essentially casts material such as appears in Chapter 1 in a 
formal mathematical mold, but the real theory does not begin until 
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Chapter 4. Thus 1, 3, and a sketchy reading of 2 might be a good chapter 
order for casual acquaintance. 

The reader interested in military applications can turn to Chapter 11, 
either at once or after the foregoing prelude. The questions of what can 
be, should be, and may be done to attain military utility fill the early pages. 
The later ones, which contain specific illustrations, require the technology 
of the text for their understanding; the reader can stop when he gets there. 


Possibly the lack of existence and uniqueness theorems! will seem a 
heresy to some. The emphasis on the specific problem, although counter 
to current mathematical trends, I feel is good and, in this case, fitting. 
Without it, it is hard to see how the innovations of the theory could have 
come to light. Besides, the very opulence of their diversity would seem to 
preclude the above types of theorems, for such would run to unwieldy 
lengths were they to cover all cases. The applications themselves have 
become more diverse than I had at first dreamed, as the reader can discover 
by skimming the pages. 

Between its first publication—the Rand Reports [1],? revised versions 
of which now constitute Chapters 1, 2, 3, and 4—there was a lapse of 
several years when I had no opportunity of giving differential games the 
concerted attention it required. 

An exceptional resumption occurred when I was on the staff of the 
Hughes Aircraft Company. The theory of collision avoidance between 
aircraft and ships is much more recondite than the unitiated might suspect. 
An investigation, spurred by a series of headlined catastrophes, revealed 
an unexcepted and elegant liaison with differential games. With coop- 
eration rather than conflict between the two players, collision avoidance 
problems cohere to the same mathematical principals as games, pro- 
viding “maximax” replaces minimax. Lack of space precludes the 
subject from the ensuing text; a separate publication will follow. 

As far as I was aware when writing it, this work was an original con- 
ception. But unavoidable delays in publication have possibly dimmed 
some of the sheen of its novelty. As has happened often before in the 
history of science, at the proper era the same concepts arise simultaneously 
and independently from widespread investigators. This work was largely 
a solitary task, and I was unaware of contemporary developments by 
others. 

In fact, it was just a few days after the completion of this manuscript 
(in March 1963) that I first saw the book [2] by Pontryagin and others, 


* The alternative used I have called the verification theorem. Its statement, proof, 
and a demonstration of its use are in Chapter 4. 
? Bracketed numbers refer to reference list at the end of the book. 
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which deals with minimizing problems through the same basic devices 
as presented herein. The technique could be classed primarily as that of 
one-player differential games. 

In his dissertation [3] of 1961, D. L. Kelendzeridze extended this 
technique to two players, and so to some extent his work tallies with 
mine. 

Besides these Soviet authors, the American contributions [4] deal 
largely with the logical foundations of the subject. Berkovitz applies 
the calculus of variations to the strategy of one player, the opponent’s 
being temporarily fixed. Fleming conceives a continuous strategy as 
lying between two discrete ones. These interesting devices of mathematical 
rigor appeared too late for their due incorporation in the present work. 

However, another native theory, roughly contemporaneous with this 
one, sounds so distinct that, while writing this book, I did not suspect 
an affinity. As developed by LaSalle and others [5], control theory is 
tantamount to that of one-player differential games and thus is largely a 
special case of the latter. I altered two terms of my Rand Reports to the 
present state and control variables in accord with control usage. The 
switching surfaces of this theory are similar to the singular surfaces of 
differential games. The question of controllability—what states can be 
reached from a given starting one—is essentially a specialization of 
differential games of kind (Chapters 8 and 9). Thus each science may 
enrich the other: control problems can be extended to games by adjoining 
an opposing player; the ideas of this book are applicable to control 
material by suppressing one player. 

However, I have retained, from the early, genetic problems which 
suggested them, the names Pursuer and Evader for the two players, 
whatever be the nature of the game. An ensuing drawback has been the 
occasional impression of readers that the subject entails pursuit games 
exclusively. 

Without the allocation of time for the task by my then employers, the 
Institute for Defense Analyses, this volume would not have been com- 
pleted. A vital part of my huge debt to this organization is owed in 
particular to Professor Bernard Koopman, now Director of Research, 
for his recognition of whatever value this work may have, both as a 
military tool and a mathematical theory. His willingness to accept a 
new idea, despite its unconventionality, is a virtue essential to our nation 
today. 

To my present employer, the Center for Naval Analyses, my gratitude 
is due for first bringing the work to published status as Differential Games, 
Research Contribution No. 1, on 3 December 1963 and for their tolerance 
in granting the time for final revisions. 
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My thanks are due to Haig Kafafian of the Center for Naval Analyses 
for his beneficial advice. To Professor Clifford Marshall, who, acting as 
referee for SIAM, offered valuable suggestions with as much understanding 
as an author could wish, and to Professors Harry Hochstadt and R. F. 
Drenick of this organization, for their enthusiastic cooperation, my deep 
appreciation. Also, my thanks to the many typists who coped with my 
symbolism, especially to those such as Mrs. Katherine Gibbon who 
mastered it. 


é Rurus Isaacs 
Washington D.C. 


October, 1964 
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Figure 1.9.2 
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mathematical model at no time gives P the right to assume that £ will 
persevere on L. 

Suppose E remains oblivious of the predatory P until 2 time units have 
elapsed. Alerted then, be belatedly assumes his optimal strategy, leaving 
Land fleeing directly away from P. These alternatives are shown on (c) 
and (d), capture occurring at X’ with payoffs 9.3 and 10, respectively. 
The former figure is the best that P can attain if he exploits all the knowledge 
at his disposal and no more. 

The same situation prevails throughout the play depicted at (c). At 
each instant P acts as if he will face optimal opposition in the future. 
This trait of an optimal strategy applies to both players and in all pursuit 
games as we shall formulate them. 

If E is known to be unproficient at detecting pursuers and we wish to 
take this into account, the thing to do is formulate a new game. We 
might, say, estimate a probability distribution for the time when Æ is 
alerted to P’s presence. In the new game the payoff would be the expected 
value of the capture time and P, presumably, will have a new optimal 
strategy. 

On the other hand, when P notes E’s oblivious course along L, is he to 
construe it to mean that E has no detection or evasive capacity at all? 
If so, of course, the collision course strategy of (d) is the best. Again a 
revised game might be constructed incorporating an estimated probability 
that E is impervious to P’s presence. But such seems a jejune approach; 
this is simply a case where a human pilot’s judgment may excel a formal 
strategy. 


Example 1.9.2.!5 Guarding a target. Both P and Æ travel with the same 
speed with simple motion. The motive of P is to guard a target C, which 
we take as an area in the plane, from an attack by E. The payoff is the 
distance from C to the point where capture occurs. We take matters to be 
as in Figure 1.9.2a, where P and E denote starting positions. 


The optimal strategies are found thus: Draw the perpendicular bi- 
sector of PE. Any point in the half-plane above this line can be reached 
by E prior to P, and this property fails in the lower half-plane. Clearly Æ 
should head for the best of his accessible points. Let D be the point of the 
bisector nearest C. The optimal strategies for both players decree that 
they travel toward D. Capture occurs there, and the length of the dotted 
line is the Value of the game. 

Let us see what happens if P plays optimally, but E does not; say he 
decides to traverse the line L of Figure 1.9.2b. Always P heads for the 
nearest point to C on the perpendicular bisector, now drawn relative to the 


15 This is the problem posed on page 10. 
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current positions of P and E. Some typical positions of this point are 
marked D, D’, D’,.... Note the increase in the lengths of the dotted 
segments. They represent the progressive penalty E pays for his poor 
strategy. Each length is the payoff were E at the corresponding instant 
to revert to optimal play and hence is the best that E can hope to recoup. 

As P aims at the moving point D, (D’, D”, ...) he describes a curved 
path’* until D reaches L. From then on D remains fixed, both players 
move straight, and, in fact, the play is optimal on both sides. Capture 
occurs at x. 

The same discussion of unwarranted prognosis, given for Example 
1.9.1, applies here. If P had advance assurance that E would never leave 
L, his best policy would be to travel straight to Y (where the bisector 
meets L). As it is, he swerves to the right before capture. He knows that 
when Æ is near his starting position, the upper right part of C is most 
vulnerable and he moves to protect it; as E advances, the danger relents 
and P moves‘in for the capture. 

At (c) we see P playing nonoptimally, traversing a line L. Now E 
should always head toward the moving point D. The dotted segments 
shrink; when Æ is at A, D actually reaches C. From then on, E moves 
straight toward this point. He actually reaches C; nothing P can do now 
will stop him. (P is at B when E is at A.) 


1.10. A PERSPECTIVE ON PRECISION 


Although it may be a shock to some mathematicians, to others perhaps 
a heresy, this work contains no existence theorems. In fact, theorems and 
lemmas themselves appear on these pages with a much lower density than 
iscommon. A sketch of the history of the subject’s growth explains why. 

From the outset we were interested in delineating and working with as 
large a class of problems as we could that would satisfy two requirements: 
they would have some kinship with reality and they could be solved. Thus 
our goal would be the obtaining of answers rather than the framing of 
theorems. 

The sought kinship did not always turn out to be severely utilitarian. 
The ideas themselves, as they were gradually revealed, tended to dictate the 
course of further researches. The homicidal chauffeur game is typical. 
We can hardly call it a problem in applied mathematics and pinpoint its 
particular application. Yet when we wish to explore the domain of 
pursuit and evasion under various type kinematic constraints, bounded 
radius of curvature seems inevitable. To start, we so restricted one craft; 


16 The reader is invited to find this path for himself by means of a stepwise, geometric, 
approximate construction. 
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hence the foregoing game. It points the way to the problem of two craft 
with curvature limitations, the “game of two cars” (Section 9.2), which, 
although the innovations of principle are manageable, entails calcula- 
tions difficult to perform by elementary means. Yet this problem is just 
what is needed in certain applications, such as that of collision avoidance 
between moving craft, which lack of space forced us to omit from this 
volume. 

But let us see what happened when such problems were attacked with 
their answers as quarry. The standard scheme of differential equations, as 
will be expounded in Chapter 4, was the first significant result to emerge. 
There were a number of intermediate stages, of course, where we were 
often guided by discrete models, such as those of Chapter 3. 

But it rapidly became forcefully evident that in many cases the differential 
equations alone hardly sufficed. Various kinds of special or singular 
behavior were often of dominating importance. In the playing space @, 
these special phenomena most commonly occurred on surfaces. A “sur- 
face” here, and in the future, means an (n — 1)-dimensional manifold in 
a n-dimensional space. Besides these singular surfaces, there can also 
exist such manifolds of lower dimension, but we have largely neglected 
the latter both because there had to be some limits to our efforts and the 
former seemed the more interesting. This because they are capable and 
usually do partition & into separate regions. 

The sorts of singular surfaces that come to light proliferated and each 
species seemed to have its own distinct theory. A classification scheme 
will be explained in Chapter 6, but it merely catalogues possible types 
rather than develope a synoptic theory. 

First came barriers (Chapter 8) and universal surfaces (Chapter 7) and 
then others. Their special yet diverse theories will occupy many of the 
later chapters. But each first arose in a particular problem; such was 
always the seed of more general ideas. There were several occasions when 
we thought the types, for practical purposes, exhausted. But subsequent 
problems incurred further novelty. At present we do not know how much 
terrain there is yet to be explored. 

Thus the general view of the typical solution of a differential game seems 
first that the playing space is cut by a number of diverse singular surfaces 
which subdivide it into a number of component regions. Within each 
component, the solution may not exist at all, but if it does, it satisfies 
certain differential equations with boundary conditions picked up at the 
bordering singular surfaces. The optimal paths—such is the route of the 
descriptive point x in the space when both players act optimally—when 
they exist with a reasonable claim to uniqueness, may have sharp corners 
when (and if) they encounter a singular surface. Besides all this, it may be 
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the singular surfaces themselves. 

With this disparate congeries of singular behavior, it is hard to see how 
there could be an existence theorem that would embrace all possibilities, 
Any that was adequate in principle would probably require such a 
colossally lengthy hypothesis as to render it pedantic in practice. 

Thus we have dispensed with attempts to frame such a theorem and 
substituted another idea which we feel is better suited to the ends of this 
subject. 

These pages will contain a fairly unified technique for getting what, for 
the moment, let us call formal solutions. A large number of examples 
will exhibit the process in action. The problem before us thus becomes 
that of proving that for the formal solutions are, in some reasonable 
sense, actual solutions. 

Two items are required—a satisfactory precise definition of a solution 
in this “reasonable sense” and a technique for showing that formal 
solutions conform to it. The first is accomplished through the concept of 
a K-strategy to be defined in Chapter 2, which also contains what is termed 
the verification theorem. Judicial use of the latter, often several times in 
the same problem, can be used to show that for any particular example 
for which we have found the formal solution, the latter can be converted 
into the essential constituent of a meaningful, K-strategic solution. 

This approach, although unorthodox, seems an appropriate one for the 
present subject. Of course, we cannot and do not make any general 
claims as to the the universal existence of solutions of differential games. In 
an earlier draft, the opening chapters contained several examples displaying 
various types of nonexistence, but these pathological examples have since 
been expunged in favor of the more positive cases that can actually be 
solved, And this task is our real goal. 


1.11. A PERSPECTIVE ON PROGRESS 


The foregoing goal, actually seeking the solutions of particular examples, 
has proved rewarding. Phenomena were encountered that, as far as we 
know, were unprecedented. Sometimes these were extraordinarily baffling, 
even the very nature of the sought solution being an enigma. We mention 
a few explicit instances. 

The “swerve” maneuver in the homicidal chauffeur game possesses 
convincing heuristic evidence of its existence; yet what of its quantitative 
features? How much should the pursuer first turn and how far go straight? 
In what direction should the evader follow and for how long? The 


[1.11] § 
that some component regions contain singular manifolds of smaller f 
dimension than surfaces, and indeed such manifolds can be imbedded in 3 
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resolution of these questions came only with the discovery of what we call 
equivocal surfaces (see Chapter 10) and nothing like them can exist in 
one-player games or the calculus of variations. 

For a long time the most baffling problem of all occurred in the iso- 
tropic rocket game, which first makes its appearance in Example 5.3. 
It differs from the homicidal chauffeur game only in that the pursuer 
navigates now by controlling the direction of a driving thrust of fixed 
magnitude, but this difference has very marked effects. The hard question 
was the game of kind—the conditions under which the pursuer could 
always capture the evader as distinct from those where the latter could 
always escape. It seemed intuitively undeniable that the second possibility 
would hold if the fixed parameters (thrust magnitude, evader’s speed, etc.) 
sufficiently favored the evader. As in the homicidal chauffeur game, E 
could sidestep whenever threatened by a faster but less agile pursuer. 

Now the playing space & can here be reduced to three dimensions. The 
set of points in it for which such sidestepping is possible appeared to be 
externally bounded by a surface resembling a semi-infinite, tapering tent 
(for details and drawings, see Chapter 9). There was an interpretation of 
this tent in terms of sidestepping so natural that it was hard to doubt its 
correctness. 

Yet it was open at one end! It did not separate the space. One of two 
things must be true. Either there were paths which connected the two 
sides of the tent through the open end, and such implied an outré strategy 
either for the pursuer to circumvent the sidestepping or for the evader to 
escape when he was in front of a faster pursuer a miniscule distance from 
its nose, or there was a way of sealing the open end of tent. The first 
alternative seeming implausible, we made attempt after attempt at the 
second. 

Only a special class of surfaces are eligible to be seals. It seemed at 
first impossible to pass one through the boundaries of the tent opening. 
Finally, an altogether different problem—the deadline game of Example 
9,5.2—suggested the answer. It is what we have called an envelope 
barrier (Section 8.5III). It embodies the remarkable feature of intransient 
vulnerability, for it is comprised of paths where the evader sidesteps and 
then, instead of leaping away to safety, he must remain at the very border- 
line of capture throughout a positive interval of time! 

With the unfolding of such ideas, we had more tools and the gamut of 
solvable problems grew. But what are the limits? Rather late, and un- 
expectedly, still another of these seemingly baffling novelties appeared. 
Superficially the problem seemed of childish simplicity; in fact, it was 
coined as an elementary example to illustrate another point. It has been 
dubbed obstacle tag and will be found in Chapter 6. We bequeath this 
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innocuous appearing game, which seems to defy all our present methods, 
as a problem to the reader. 

We can list a few more general tasks still to be done. 

The criteria of universal surfaces (Chapter 7) have been found only 
when the dimension of & does not exceed four. Not only should this be 
carried further, but the case where n = 4 could very likely be improved. 

Singular manifolds of smaller dimension than surfaces should be 
studied. 

But, most important of all, the theory should be freed from the require- 
ment of full information. Some of the difficulties and possibilities are 
discussed in Chapter 12. 


1.12. ON READING THIS BOOK 


We have stressed the value of examples as illuminants of broader ideas. 
There will be many in the ensuing text, which will result in pages filled with 
formal calculations. 

There will naturally be a tendency for the reader to gloss over these 
rapidly and to get on to new concepts. Even the author has done so when 
reviewing a part of the manuscript that had been drawer bound for some 
months. But the examples are too intrinsic a part of these researches for 
omission. How should they be approached ? 

First, we have adopted a standard scheme, which will recur often, of 
writing the basic differential equations which will be explained in Chapter 
4. There was a choice to be made here between the economy of eschewing 
all repetition and the clarity of precise restatement, and we felt it advisable 
to favor the latter. The labor of writing a few extra lines is not great 
compared to that required to reconstruct an obscure meaning; actual 
redundancy results only very seldom. As one acquires facility in treating 
problems, he can and should make his own abridgements, but they are 
best avoided in a public exposition. 

Second, despite the standardized format, the examples are not routine. 
Each has significant, unobvious, and nontrivial features of its own, which 
are apprehended only by plunging into the analysis. 

Third, we suggest that certain readers, instead of following the steps of 
the text, may wish to work out the examples on their own scratch pads. 
They can use as guideposts the author’s experience in having already 
been over the ground to garner hints from the text. The gain is of both 
instruction and enjoyment. 





CHAPTER 2 


Definitions, Formulation, and Assumptions 


The concepts entailed in a differential game are translated into the 
mathematical vernacular and thus made precise. Alternatives are moti- 
vated and illustrated by examples. Several assumptions that will prove 
useful in our later analysis are stated and justified.? 

Throughout we deal with games of perfect information. That is, at all 
times each player knows the values of the state variables. 


2.1. THE KINEMATIC SITUATION 


Our theatre of operations is &, a region in Euclidean n-space and its 
boundary. This boundary is to consist of pieces of certain surfaces (we 
mean by surface an (n — 1)-dimensional manifold’). We think of a 
particular point x = {a,,...,2,} to be in motion in @, its path being 
governed by what we shall call the kinematic equations, abbreviated KE: 


s Eng Pry sss Bas Piss +s Weds j=1,...,n (2.1.1) 


È; = f(t T 


or more briefly 


k = f(x, $, y). (2.1.1) 


The functions fare given; we suppose them to be of a simple character, 
and in the sequel we shall not hesitate to speak of any partial derivative 
(of any order) of the f; that we have occasion to need. We term ¢ and y 
the control variables. They are at all times one each under the control of a 


1 Originally Rand Report RM-1399 (30 November 1954). Certain additions and 
improvements have been made and several pathological examples deleted. 
2 Supposed piecewise smooth. 
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player. Thus the motion of x is to be thought of as influenced by the wills 4 


of two individuals. If they seek conflicting objectives—and only such 


cases are of interest*—the situation assumes something of the nature of a | 


game. As suggested by game theory, we will speak of a particular x in & as 


a position or state; we call the z,,..., x, the state variables in that they 3 


describe this state; the two individuals are the players. 


The x, are descriptive in the following sense. If a play of a differential 3 ; 
game is halted before completion, the values of z}, ... , 2, at the time of $ 


interruption supply all the data needed to resume the partie. We mean 
that if a new partie is commenced starting with these x,, it will be tanta- 


mount to the part of the original that would have occurred after the | 


interruption. 

In particular, the values of x, at the outset supply starting data. Thus 
when we use the term game, we are not speaking of a single game but of a 
collection. There is a distinct game emanating from each point of €. 

In general, the ¢ and y are (individually) subject to certain constraints 
which depend at most on x, the typical form being: a,(x) < 4, < b,(x). 
It will always be understood that we are interested only in ¢ and y which 
satisfy the constraints, and explicit mention of them in the future will be 
considered unnecessary. 

With y fixed and x fixed in &, the set of vectors f(x, $, y) for all ġ will 
be called a vectogram or a ¢-vectogram (similarly for a p-vectogram). A 
full vectogram allows both ¢ and y to take all values. 

For example, planar simple motion is depicted by a circular vectogram 
of fixed radius (= the speed) at each point. Such a ¢-vectogram* is shown 
at Figure 2.1.1a. At (b) is shown a simple prototype of a full vectogram 
with n = 3 and ¢, p having one component each. 

Presupposing the numerical payoffs to be introduced in Section 2.4, we 
name the players: 

P, controlling ¢ and minimizing. 


E, controlling y and maximizing. 


The names relate to the Pursuer and Evader of pursuit. games. Although 
such games were the seeds of the present theory, its scope now embraces a 
wide range of phenomena, as the subsequent examples will show. Because 
we have retained the designations above, let not the cursory reader infer 
that contests of pursuit and evasion are synonomous with differential 
games. But pursuit games, especially in those early chapters, are fine 
vehicles for illustrating points of the theory in general. 


è There can be exceptions, such as in the theory of collision avoidance where the 
participants cooperate. 
“When a control variable has but one component, we shall omit the subscript. 





[2.1] 





THE KINEMATIC SITUATION 27 





The names, P and E for the players in general types of problems have a 
Jess sterile ring than such nomenclatures as Player I and Player II, or Red 
and Blue. Our choice gives the players individualities—personalities, if 
one prefers—without sacrificing a certain symmetry of roles which is, in a 
sense, the essence of game theory, as opposed to one-player problems. 

The obvious mnemonic as to roles is in terms of pursuit games with 
time of capture as payoff: P minimizes, E maximizes. me 

When working problems we shall often use more descriptive letters 
than £ %»,-. . for the state variables. Standard coordinates such as (x, y) 
or (r, 9), for example, can designate a point in the plane and subscripts 
may be added if there are several points. Or we will use initial or other 
suggestive letters to denote entities, such as men, munitions, and time. 


Example 2.1.1. Planar pursuit games with simple motion. In both Examples 
1.9.1 and 1.9.2, if we let (21, y1) be the coordinates of P and as (£2, Y2) those 
of E, and w,, Wa their respective speeds, the KE can be written 

ż = w, sind 

jy = w, cos $ 


= w, sin y 


R 
i} 
| 


Yo = Wz COS Y. 
Example 2.1.2. The homicidal chauffeur game. A natural formulation of 
the problem requires five state variables: two coordinates each to specify 
the position of P and E and one more to specify P’s flight direction. They 





(a) (b) 


Figure 2.1.1 
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Figure 2.1.2 


can be taken as 2, Y1, ta, Y2, 9 as shown in Figure 2.1.2. Such specify 
completely and uniquely determine the position at any instant of play. 
Turning to the control variables, that of E is the simpler. One, y, is needed 
to describe his flight direction as shown. For that of P, draw the line 
C’PC perpendicular to P’s velocity vector. The volition of P allows him to 
choose his current center of curvature at any point, such as C}, on this 
line with the open segment C’C excluded (|C’P| = |PC| = R). Typically, 
we define his one control variable ¢ as R/|PC,| so that -1<¢ < 1. 
Then the KE (kinematic equations) are 


z, = w sin 0 
Jı = w, cos 0 
È, = wa Sin y 
Üa = W2 COS Y 


Wi 
b= tg, 


A player may have more or less control of his present and future, but no 
one can affect the past. Thus we interpret the left sides of each KE as a 
forward time derivative. 


2.2. THE REALISTIC AND REDUCED SPACE 


When we construct a model from a physical prototype, the set of state 
variables will generally be such as to convey a direct and forthright 
description of the situation. Their number n, the dimension of &, however, 
may be larger than necessary. Often by judiciously picking a less direct 
set of coordinates n may be lowered. When such has been done, we will 
speak of the reduced space. We will use the symbol & for it as well as for 


$ 
i 

p 
F 
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[2.2] 
the original, which we shall call the realistic space.5 In either case, in 
general discussion, n will be the dimension. 

The advantages of a reduced space, with its smaller number of state 
variables are less redundancy and required writing. Also, if n can be made 
3 or less, the convenience of geometric visualization is sometimes a great 
aid in intricate situations. But there are arguments in favor of the 
realistic space. 

The KE, although more numerous, are sometimes much simpler. If 
the problem involves moving craft, their paths in the realistic space are 
what they are physically; in a reduced &, a very simple such motion, such 
as in a straight line at constant speed, may appear almost unrecognizably 


complex. 


Example 2.2.1. If, in Example 1.9.2, the target area is a half-plane, say 
that below the y-axis, we can employ a reduced space with n = 3 instead 
of 4 as in Example 2.1.1. 

If we put x = 2 — 2, it is clear that knowledge of z, y1, Ya suffices to 
describe a state. The KE then become (In Example 1.9.2 the speeds were 
equal; if such is desired here put w, = we): 


č = w sin d — w sin y 


jı = w, cos $ 


Yo = Wg COS Y. 


Exercise 2.2.1. In the preceding example show that if the target area is 
circular, say centered at the origin and of radius R, then n can similarly be 
reduced to 3 and write the corresponding KE. 


Exercise 2.2.2. Show that in Example 1.9.1 there is a reduced space with 
n=1, 


Observe that such reductions may not be possible if the realistic space 
should lose its homogeneity, say if the speeds were functions of x, y. 


Example 2.2.2. The homicidal chauffeur game. A reduced & can lower n 
from 5 to 2. Let us think of a map of the realistic space (the “parking lot”) 
as affixed to the car P. We can use the coordinates x, y of E’s position, 
which will be x on this map, as the sole state variables. The y-axis is 
always to be in the direction of P’s velocity vector. 


5 We prefer to leave these definitions somewhat flexible rather than tie ourselves 
down by stringent requirements. The usages in subsequent examples will render the 
idea clear. We have not coped with the general problem of ascertaining the minimal 
possible n. 
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Figure 2.2.1 


As in Figure 2.2.1, let P choose his center of curvature at C = (R/¢, 0) 
and let d be the distance CE. Then P’s rotation about C is tantamount to 
a rotation of x about C in the opposite direction with the same angular 
speed. Thus x moves with speed w; (d$/R) in a direction perpendicular to 
CE. Its velocity components are obtained by multiplying the speed by 
—y/d and (x — R/¢)/d. Thus the KE are 


= -Ty + wesin yp" 


y= Tag — Wi + we cos y, -1<¢<1. 

We take @ as the origin centered circle of radius / and & as the portion 
of the x, y-plane exterior to @. (See the next section.) Note that if E 
travels straight but P deviously, y will be an involved function of the time. 
Such is typical of the disadvantage of a reduced space. 


2.3. TERMINATION OF THE GAME 


There is a surface Q, called the terminal surface, which is part of the 
boundary of £. When x reaches @, the game is over. 

We take this form of termination as part of the canonical definition but 
feel obliged to defend our motivation. Why a surface? Why part of the 
boundary of &? 

The termination of a pursuit game usually is capture, which at first 
glance we might take to mean the coincidence of P, with E.” If the x; are 


® Not the same as the old y, of course. 

7 Simple pursuit games involve the motion of two points, one pursuing the other, as 
explained in the ensuing paragraph. We shall use P and E for the names of these points 
as well as for the names of the controlling players. 
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the totality of variables descriptive of the positions of both P and Z, the 
subset of & corresponding to capture would be of dimension <n — 1. 
We reject this definition of capture on two grounds. 

First, it is unrealistic. In applications, the point P or E will be some 
fixed spot on a missile (aircraft, ship, torpedo, etc.) of tangible bulk, 
intended to serve as an index of the missile’s location. On these grounds 
alone, P and E will never coincide; but in tactical situations often all that 
is required for capture is less even than physical contact—just a certain 
proximity. Thus a more reasonable criterion of capture is, say, to specify 
a positive / and capture occurs when the distance P to Fis 7.8 The set of all 
capture positions is specified by one equation and thus constitutes a 
surface in &. 

The second ground applies generally. The technique we shall use entails 
differential equations. A surface provides just the number of dimensions 
needed for initial conditions so as to obtain unique solutions. A lesser 
dimensional manifold generally gives rise to singular points of the 
solutions. (In Chapter 6 are some examples of games which are pathologi- 
cal because of this dimensional deficiency.) Should we be presented with a 
game with a terminal set @’ of too small a dimension, we amend it by 
using the boundary of a é-neighborhood of @’ for a terminal surface. 
If desirable, we can investigate the limiting situation as 6 > 0.9 


Suppose that, in the formulation of a game fresh from the physical 
situation, the surface & is not on the boundary of & but is interior to &. 
Locally it separates & or, relatedly, @ has two “sides.” Often we will wish 
to count as termination only the cases where x reaches @ from a particular 
side. For example, let us return to the above pursuit game, and suppose we 
started from a position where the distance P to E < I. Clearly we would 
not want to consider a subsequent occurrence of |PE| = l as capture. 
What we do here is exclude all positions with |PE| < / from &. Then € 
will be part of the boundary. 

However, there may be cases where @ is desirably in the interior of &. 
We will then distinguish between approaches of x to the two sides of @. 
We can think of & as “slit” along @ and @ itself as two-sheeted. Thus, in 
a sense, we have restored to @ its role of boundary. 

What shall we do if x never reaches @? A reasonable and quite practical 
thing is to supply a stop rule. That is, we select some large value T of time 
and decree that the game is over should T elapse. We can bring this 
situation into the canonical picture by introducing time as a new state 

8 There can, of course be cases with “capture regions” of shapes other than circular. 

? When working problems by simple geometric methods, such as the examples in 
Chapter 1, the coincidence of P and E seems a convenient theoretical criterion of capture. 
See Sections 6.7 and 6.8. 
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variable, x,,;. We enlarge the kinematic equations with 4,,, = 1 and 3 
take for the new £ the direct product of the old by [0, T]. The new @ is 3 
the direct product of the old by [0, T] as well as the part of zı =T 4 
bounding the new &. We only consider plays of the enlarged game which 


start from an x with z,,, = 0. 


As @ is a surface, an (n — 1)-dimensional manifold, we can represent 3 F 


it by n — 1 parameters. Our standard general way of so doing is 


x; = hfs, © <- 5 Sn) = AS), Pe his, m (2.3.1) 3 


We shall assume these functions to be differentiable. In our problems, | 
at worst @ will be piecewise smooth; we treat each piece with a separate 3 


representation (2.3.1). 


2.4. THE PAYOFF 


The numerical quantity which the players strive to maximize and 4 
minimize in games of degree can assume a variety of guises. We prefer to # 


take as the standard form of the payoff 
foa $, y) dt + H(s). (2.4.1) 


Here the function G is qualified as are the f; (as to having partial de- 
rivatives, etc.). The time integral extends over the path in & traversed by 
x during a partie; its lower limit (we could call it t = 0) refers to the 
starting point in @; its upper limit is the time of termination—when x 
reaches @. 

The function H is a smooth one defined on @. For any partie, the 
second term of (2.4.1) is the value of H at the point of termination— 
where x meets @ as play ends. 

Of course, if both H and G were 0 we would have a vacuous situation; 
we exclude it. If H = 0, we will say the game has an integral payoff; 
if G = 0, a terminal payoff. Almost all practical examples are of these 
two types. 

As instances, pursuit games with time of capture as the payoff have 
integral payoffs with G = 1. Example 1.9.2 has a terminal payoff: the 
distance from E to the target when capture occurs. 

For certain theoretical purposes games with terminal payoffs are 
advantageous and we shall then make use of 


THEOREM 2.4.1. A game with a payoff of the form (2.4.1), with G # 0, 
can be replaced by an equivalent one with terminal payoff. 
Proof. Primed symbols refer to the new equivalent game; unprimed 
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nes to the original. For & we take the direct product 


€xXL 






the L meaning the doubly infinite interval (— œ, œ); it is to be the 
range of a new state variable denoted by z,,,. Similarly, 


C=EXL 
and the parametrization of @” will be (2.3.1) together with 
(2.4.2) 


Trt = Sw 


To the old KE we adjoin 
Entr = G(X, $, Y). (2.4.3) 


The payoff will be terminal with 
H(s') = H(s) + Spy. (2.4.4) 


Now let us consider a partie of the new game starting from x°, with 
components z,°, and terminating at a point s’ of @’. As ~,,, is not involved 
in the first n kinematic equations, if we project the path on @, it will 
correspond to one resulting from a partie of the old game. Conversely, 
any partie of the old game corresponds to one of the new. For %,..., 2a 
as well as ¢ and y, will then be known functions of t; they can be inserted 
into (2.4.3), which is then integrated with the condition x,,,(0) = 2°,,. 

What is the payoff of a partie of the new game? It is given by (2.4.4), 
where s’ is the termination point on @’, consisting of s on @ and s,. By 
integrating (2.4.3) from t = 0 (at x°) to its final value (at s’) we have, 


using (2.4.2), 
stam | GO, fw) at 


the latter integral extending over the path in &” or, what is the same thing 
(«,4, İS not in the integrand), its projection on @. Substituting in (2.4.4) 
we have 
Payoff = H(s) + fo% $, y) dt + tha 


If we confine ourselves to starting points with 2°, = 0, then the payoff 
will be exactly that of the original game. 

Note that no essential generality is lost by this confinement. As pı 
appears on the right in no KE, all paths in & which differ in starting 
positions only in z? , , are translations of the same one in the z,,,, direction. 


Other types of payoffs are subsumable in the form (2.4.1). 
Suppose ¢ (time) effectively appears among the arguments of the fj, of G, 
or even of H. In the latter case, the payoff is a function of the time of 
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termination as well as the place. Then we adjoin #,,, = 1 to the kinematic 3 
equations, take the new @ and @ as the direct products of the old by the 4 
full interval (— 00, 00) of x,,, and replace the argument ¢ in f,, G or H by 4 
%n4,;- When the revised game has been analysed we discard all Starting : 
points except those with x,,, = 0. 
There are applications with payoffs 


Vital for our work is the 
MINIMAX ASSUMPTION. For all u and all x in & 
min max Q = max min Q. 

$ Y Y $ 
In all applications encountered up to the time of writing, each f; and G 
P have been separable, that is, of the form: function independent of y + 
f G(x, $, y) dt unction independent of ¢. In such cases the minimax assumption 

9 4 obviously holds. 

where T is some prescribed positive value of the time (t = 0 means the 
starting time). Such a T is essentially a state variable. We adjoin to the } 
KE : 


tj 


2,5. GAMES OF KIND AND GAMES OF DEGREE 


When we speak of a game of degree, we mean one with a continuum of 
possible payoffs such as in the last section. A game of kind has finitely 
many, usually only two, the outcome of the game depending on whether 
or not one of the players can achieve some objective. For example, in a 
pursuit game the objective might be capture; in a battle game, complete 
extermination of the opponent. 

If a stop rule is imposed, a game of kind becomes one with terminal 
payoff for which H assumes only finitely many values. The game falls 
within our compass and no special treatment is required. Nevertheless, 
it is often possible and desirable to imbed the game of kind within one of 
degree and deal with the latter. 

The solution to a game of kind may be tremendously indeterminate. 
It results in the division of @ into two (sometimes more) aliquot subsets 
(some possibly null), one favorable to each player. If the starting point 
lies in a player’s set, he can attain his objective. Then usually any strategy 


T= inn = —1 


and take the new & as the direct product of the old by [0, œ). For © we 3 
take that part of the boundary of the new & where x,,, = 0. We play the 
revised game with an integral payoff with integrand G. We utilize only $ 
starting points with x,,, = the prescribed value of T. 
Suppose we are given a function K(x) defined in &, and the payoff is to 4 
be the value of K(x) at the end of a prescribed time T. We treat this case ` 
similarly to the preceding but use a terminal payoff with H = K. i 
Another type of payoff which can be reduced to the standard form, not 4 
always but at least in simple cases, is as follows: Let K(x) be given in &, | 
The payoff will be the minimum of K(x) which occurs during the play. $ 
Example: In a pursuit game, how close can the pursuer get to the evader? @ 
Let &, be that subset of & in which £ can cause K(x) to increase whatever $ 
P may do. That is @, is the set of x for which 





max min X K, fi(x, $, y) > 0.” (2.4.5) 4 is optimal for him as long as it lets him remain in his set, whereas any 
v $ 3 1 strategy at all is optimal for his opponent. These ideas will be developed 
Let @ be the boundary of @,. It is clear that if a minimum of K occurs at “} at length in Chapter 8. 
all, against optimal opposition from Æ, it will occur on @.!" Thus we may | Let us take two species of games of kind: 


reduce matters to a terminal payoff with H being the value of K on %. 
The reader can easily construct examples in which P can achieve low 
minima only by causing x to enter &, and leave it again; in such cases the 
idea just given will present its difficulties. 


1. A pursuit game with capture as the objective. 

2. The same game with the objective capture before a stipulated time T. 
Such would be the case, say, if P were an interceptor missile with a 
limited fuel supply. 


i yen 


For a vector u = (4,,...,u,,) write for short 
; In both cases we would lose nothing and might gain much if the time 
Q= > uf, $, y) + G(x, $, y). : of capture is made the payoff, which is taken as oo if capture does not 
: occur. We can then expect definite optimal strategies instead of a sprawling 


: : : d : ; ; class delineated only by inequalities. In case 1, the strategy will not only 
1 We are supposing all given functions continuous, differentiable, etc. In general, ` 


ĉ, will be open and @ a surface. In fact, @ will be defined by (2.4.5) with > replaced : instruct Pas to how to capture but will show him how to do it as quickly 
by =. : as possible. Similarly, it tells Æ how to delay it. If we have case 2, we need 


1° Unspecified limits of summations will always be 1 and n. 
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only look at the Value and see whether or not it exceeds T. We will have 
solved case 2 for all possible values of T at once. 

However, we do not advocate abandonment of the game of kind; inj 
fact, some later chapters will be devoted to them. There are cases whered 
the direct solution is much simpler than the embedding procedure j just! 
suggested and supplementary information of little value. Sometimes, too, 
a game of kind appears as a phase of a game of degree.. For example, a 
player may not be able to terminate favorably unless he first surmounts $ 
some obstacle. The question of whether this can or cannot be done may: 


constitute a game of kind whose solution is a preliminary to that of thed 


whole game. 
When we speak of a game without specifying whether it is of kind or. 
degree, it will be understood to be of the latter type. l 


2.6. STRATEGIES 


In the theory of discrete games, a strategy is defined as a set of decisiongį 
for a player, one for each position that may arise. If each player chooses ağ 
strategy, the ensuing partie, and particularly the payoff, is uniquely 


determined. 


We recognize a somewhat analogous circumstance existing here. The 
election of a decision for each possible position amounts to a player's 
choosing his control variables as functions of the state variables. If the: 
players each so select (x) and y(x) and these values are inserted in the} 
kinematic equations, the latter become differential equations. Recalling’ 
that the data of a game must include a starting value of x, we see that this $ 
value plays the role of an initial condition. Thus, under reasonable cir- $ 
cumstances, we may expect the paths—and hence the payoff—to be $ 


uniquely determined. 


As in general game theory, the Value of the game is to be the minimax | 


of the payoff. Symbolically 


V(x) = min max (payoff). 
H(X) vx) 


Here the min [max] extends over all allowable strategies 4(x), [y(x)]. 





















We are to expect that the min max equals the max min, an expectation that 4 


will be justified later on. (It depends rather strongly on the minimax 4 


assumption of Section 2.4.) 


We shall always capitalize the term Value so that we may retain literary # 


usage of the word “value.” The Value is a function V(x) of the starting 4 


point x, a function that will play a key role in our subsequent analysis. 


1? The Value of the game, a standard term in game theory. It is fitted to our context 4 


in the next section. 
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se concepts carry a measure of heuristic satisfaction; but we must 

„robe them too much with logical precision. 

t each instant during the course of a play, the players are faced with a 
vectogram. If we think of each as choosing a value of his control 

ables, the choice results in the selection of a constituent velocity vector 

ong which x will travel in the immediate future. Thus what corresponds 

“the moves in a discrete game is here a continuous and unrelenting 

ce of ¢ or y. The reader may justly protest that we are demanding 

ofthe players feats beyond human ability and of the mathematics problems 
ond rigorous analysis. We shall assuage him shortly. 

TAN attempt to define strategies in the form ($(x), y(x)) leads immediately 





to difficulties. First we require assurance that the differential equations to 


which the KE are converted are integrable. We recall that the left sides 
are to be construed as forward derivatives. Now the existence criteria 
for “forward differential equations” are much broader than those of the 


: classical theory and their limitations quite distinct, as the following 


examples will indicate. Let us take first 


#,(forward derivative) = f,(%,, %2), i=1,2 
where (fi Ja) = (1, 1) when z, < 0 

= (0, 2) when x, = 0 

= (—3, 0) when 2, > 0. 


The reader will see that this system has exactly one solution starting from 
each point in the plane. Later we will find that functions of this genre are 
no strangers to the solutions of differential games. 
Now let us take for f: 
(1, 1) when zı < 0 


(—3, 0) when 2, > 0. 


We are frustrated should we start from or arrive at a point with x, = 0. 
A theory of such differential equations now claims a number of papers,!? 
but we have not followed this possible route in our work. 
In the sequel we shall develop methods of solution. The results will 
include values of 4, y, say $(x) and (x), which we shall term optimal. 


18 H. Bilharz, Z. angew. Math. Mech. 22 (1942), 206-215, D. Bushaw, Contributions 
to the theory of nonlinear oscillations, IV, Princeton 1958; Fliigge-Lotz, Discontinuous 
automatic control, Princeton 1953; J. André and P. Seibert, Archiv d. Math. 7, 148-156, 
157-165 (1956), Comptes Rendus 245, 625-627 (1957) Boletin de ta Sociedad Matematica 
Mexicana 242-245 (1961). Solncev, Moscow. Gos. Univ. Utenye Zapaki, 48, Mat. 4, 
144-180 (1951). A more detailed exposition of the subject will appear in the “Con- 
tributions to the Theory of Nonlinear Oscillations,” Vol. V, edited by S. Lefschetz. 
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When inserted in the kinematic equations, these strategies render them at. . 
least piecewise integrable (or integrable in the sense of forward differentia] 3 
equations). Solutions (paths, the payoff, etc.) can generally be computed, 4 
and they appear to be optimal in the sense of rendering the payoff a 3 


minimax. 


But there remains a second difficulty. An assertion that, say, ¢ is | 
optimal entails knowledge of its performance when opposed by a certain 4 
class of y. What class? It must be such that the pair $, y will always lead 4 


to integrable KE and that all y representative of an actual player’s actions 
are included. 


Samuel Karlin has advanced an idea which obviates these troubles. A 4 


strategy for P is defined as both the choice of a function ¢(x), now subject 


to no restrictions (save the constraints) and a sequence o, = {fọ = 0,4, 3 
tə, ...} of increasing values of time with lim ¢; = œ. Such will be called a 4 


j l 
K-strategy. In playing it, suppose P at time ż, finds x to be at x™ (x js į 
the starting position). Then in the interlude [t,, t%+1), P holds ¢ to the 4 


constant value ¢(x‘*), 


Let a K-strategy, y(x) and o; = {tọ = 0, t’, ...} be also defined for į 
E. We subdivide time by both the ¢; and the t,’. In each subinterval both 4 
¢ and y are constant and so the KE are obviously integrable. We build the 4 
path, using as the initial values for each later interval the final x from the 3 


previous. 


Thus for each starting point and pair of K-strategies, the path of x, and 4 


consequently the payoff, is uniquely determined. We define the Value as 


the sup inf (= inf sup) of the payoff, the sup and inf ranging over the $% 
respective players’ classes of K-strategies. Such is the natural counterpart } 
of the more primitive minimax definition given earlier in this section, 


It is hard to see, in the world of actuality, how a sequence of decisions 


could be anything but discrete. Thus the K-strategies seem a step nearer } 


to reality. 


We will speak of the 4(x)[y(2)] which is a constituent of K-strategy asa $ 


tactic.4 


It is manifest that, in general, the K-strategies will not yield optimal $ 


strategies but only -optimal strategies, that is, strategies that will attain 


within e of the Value (this being done, it would seem, by increasing the § 


fineness of the temporal subdivision). 


1 The terms strategy and tactic have nothing to do with the military lexicon. The $ 
former, since its introduction by von Neumann and Morgenstern, has become standard 4 


in game theory and we use an obvious extension. We introduce the latter when speaking 


of K-strategies because an alternative word is convenient. Its usage in this book will 4 


be quite limited. 
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n we be sure that strategies subsume all the best modes of playing? 
tus, for the moment, waive subtleties and accept strategies in our earlier 
ise. Suppose a player, say E, follows the dictates of a policy S,* not a 
stegy as we have defined it. For example, Sg*, may entail a y whichis a 
ion of #,, other higher derivatives of the x, (E somehow provides for 
es where they no not exist), past values of the £; integrals over such 
t values, etc. If E pits Sg” against an optimal strategy Sp for P, 
ow do we know he will not emerge with better than the Value? ` 

We will endeavor to reply in two ways. The first is heuristic. It is based 
on the fact that the state variables are truly descriptive of the state in the 


-sense discussed in Section 2.1. 


To illustrate, let us consider in part a pursuit game in which P is a point 
moving in a plane. Let x,,z, be his coordinates. First let him have 
simple motion so that the kinematic equations are in part 

2, = w cos d, 

t = w sin dy. 
We claim that the only rational policy for E is to base his actions on x, and 
z alone. He might have them depend on, say, 2, č past values of the z; 
etc. as well, such as would be the case if he endeavored to extrapolate P’s 
future positions. But P’s velocities—according to the way we have framed 
the problem—are at all times subject to abrupt change without notice. 
It is impossible for E to rely on any prediction or indeed to derive any 
constructive knowledge from anything other than x, and 2». 

Let us now make P’s motion a bit more complicated. We suppose that 

now he regulates the acceleration ¢,, $z (subject to some bounds that we 
will not mention). The KE in part now might be 


Ly = T3 
Žo = Ly 
tz = gy 
ty = do. 


Now P can no longer abruptly switch velocities, and there are sound 
grounds for E’s basing his policy on them. But the velocities are xs, x, and 
appear among the state variables. However, the same argument as before 
shows that E could be misled if he based his decisions on, say, P’s accelera- 
tion. 

We can proceed thus, creating a chain of more and more complex types 
of motion for P and, in fact, can introduce many variants and offshoots 
along the way. In each case we single out those data of the motion on 
which it appears E can rationally rely when making his decisions, and in 
each case they appear among the state variables. 


Ce ay 
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The second reply is mathematical. Suppose P plays Sp and E plays Spi 
from some starting point. On the resulting path we will have th 
arising from Sg* defined as a function of t. We take the liberty of supposing 
that this function is piecewise continuous. Then there will be a strateg ny 
Sz for E which will agree with Sẹ* whenever a partie results in this sam 
path. Thus Æ will reap the same yield if he plays Sz or Sg* as long as 
adheres to Sp. As Sp is optimal, E cannot do better than the Value’ 


t 


2.7. CANONIZATION OF THE VECTOGRAMS 


In our science, as in other fields of analysis, we are free to make certai 
transformations on the variables involved. For example, if is a smootff 
surface, we can select the state variables so the @ lies in the surface wherg 
x, = 0. But we wish to speak here of transformations on the controff 
variables which bring the vectograms to certain canonical forms that wi 
later prove convenient. 

We can, of course, and should assume there are no redundancies: on șġ 
¢-vectogram, say, we choose coordinates (the ¢,, . . . , $1) so that to each 
velocity in the vectogram there corresponds but one set of numerical 
values. It follows, then, that A < n. k 

Another rather obvious requirement on meaningful vectograms is that 
they permit sufficient scope to the navigation they allow x in @, rather than! 
confine x to a lower dimensional subset. In this event we could reformuy 
late the problem taking for @ this subset. 


Example 2.7.1. Let n = 3, A= 1, x = 0. The KE are 


x = a(x)d + A(x), —-1<¢,<1 
with y(x) = « x 6 #0. Then, if 
y-curly = 0 


a known result of classical analysis tells us that & is covered by a family ' 
of surfaces such that everywhere the vectors « and f lie in the tangent $ 
plane. Then x must always remain in the same one of these surfaces from $ 


which it started. We can use this surface for &. 


We will say a vectogram is convex if, whenever v, . 
so does XZ _,C,v,, where C; > 0, ICG = 1. 
Less trivial than the foregoing is the 


CONVEXITY ASSUMPTION. All ¢- and y-vectograms are convex. 


Rg 


. , 0, belong to it, J 
















Should this assumption be violated, there may be no solution. We do: $ 
not reject the game but replace it by another in which the ¢- and p-vecto-: $E ~ 





grams are the convex hulls of those of the old; that is, the new vectograms l 
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Figure 2.7.1 


are the smallest convex ones which contain the old. Ifthe new game can be 


solved, its solution will supply the essential information about the old. 
An example will clarify matters. The reader will see how to apply its 


idea generally. 


Example 2.7.2. In this game y does not appear, so that it really is a 
minimizing problem rather than a game. Here £ is the part of plane above 
the curve @ of Figure 2.7.1. The vectograms are the same for all x; one is 
sketched. Let M be a high point of @; P is to start from x°, directly above 
M, and reach @ in the least time. Clearly a solution will entail a zigzag 
path arising from an alternate use of P’s two extreme velocities. There will 
be many solutions. 


Example 2.7.3. Let L be a vertical line through M. We alter matters by 
letting the vectograms preserve their form but letting them decrease in 
magnitude with the distance from L. Then clearly P does the better with 
finer zigzags which stay closer to L. There is now no solution. 

Perform the alteration described above, replacing the vectograms by 
their convex hulls. The game now has a solution: P traverses L to M. 
We see in what sense this solution is approximated by those of the unaltered 
game. Thus by replacing vectograms, when necessary, by the convex hulls 
we obviate troubles of the above type. It is easy then to interpret the 
original game. 


If $ corresponds to a member of a $-vectogram, then cd, where 0 < c < 1, 
is a convex combination asserted to be also a member by the convexity 


18 The Value however exists. 
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assumption. But it turns out that such fractional multiples are seldom 
of importance in practical problems. If the motion of a craft is involveg 
decisions as to the best direction are the real problem; optimal strategieg 
almost always dictate the maximal allowable speed. Therefore, in man 
of the problems to come, the vectograms shall not be convex in thijs 
fractional multiple sense (the reader can always make them so by introd 
ducing a new control variable like the c above, but he will find its optimal 


value to be 1), but only in the sense of closure with respect to vectors of 


distinct directions. 
Somewhat similarly we have the 


CLOSURE ASSUMPTION. All ¢- and p-vectograms are closed. 


The grounds are similar to those of the last assumption. If there were l 
convergent sequence of members of a vectogram that incurred increasingly 
favorable payoffs, technically the game might lack a solution. Therefore} 


it is wiser to include the limiting velocity, which we presume to be optimal, 


in the vectogram. Thus we circumvent difficulties of this kind. 
As before, when faced with a game with improper vectograms, here 
meaning they are not closed, we simply adjoin their closures and proceed§ 
with the analysis. q 
We may consequently assume that the lengths of the vectors in any, 
vectogram are bounded, for, if not, the closure assumption would imply! 
the existence of infinitely long vectors. If these were actually used in ang 
optimal strategy, we could soundly judge the game to be pathological or; 
trivial. If they were not, we could excise them and some neighbors from; 
the vectogram without effecting anything essential. í 
The latter two statements together imply 


We may assume all ¢- and y-vectograms are closed and bounded and] 


therefore compact. 
We come now to the useful result: 
The constraints on the control variables may be taken as constant. 


That is, for example, 


with a, b; independent of x. For we know that we may assume each} 
vectogram to be a compact, connected (because of the convexity) set, and, j 
from our general hypothesis as to the f,, these sets vary smoothly with x. 3 
We can then find a smooth mapping of these sets of vectors {¢,,...,¢} E 


into, say the unit cube of Euclidian A-space, which is also a smooth? 
function of x. Using the coordinates of the cube as new ¢; we will have} 
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1) (with 4; = 0, b; = 1, but the values do not matter). The y- 

grams can be treated similarly. 

na game with terminal payoff, as only the location of x at termination 

tters, a change of time scale makes no effective difference. We can even 

‘ce this scale change locally, varying it from point to point in &. For- 

tty this is accomplished by multiplying the right sides of the kinematic 

uations all by the same positive function of x, that is, the typical KE is 
ced b 

Pe i, = ulx) f(x, 4, 9. 

tis clear that as long as u(x) > 0, the curves of the paths are unchanged 

d hence the same strategies yield the same payoffs as before. 

"As we may assume the vectograms bounded, we can take u(x) such that 

+ each x its product with the longest member of the full vectogram is 

sounded throughout £. Thus we may assert 


In a game with terminal payoff we can arrange, without effectively 
changing the game, that the vectograms are uniformly bounded through- 
out Ê. 


9.8 A LEMMA ON CIRCULAR VECTOGRAMS 


The following simple result will have utility in many of our later 
problems. 


LEMMA 2.8.1. Let u, v be any two numbers such that 


p=vVu +> 0. 
Then 


max [min] (u cos ġ + vsin ¢) 
$ ¢ 


is furnished by $, where 
cos $ = +[-]1*,  sinġ = +[-]? 
p P 


and the max [min] itself is ; 
+[—]p. 


Proof. The quantity in round parentheses is the inner (scalar, dot) 
product of the vectors (u, v) and (cos ¢, sin $) and therefore is the pro- 
jection of (u, v) on the ray through the origin having inclination ¢ with the 
u-axis. The maximum then occurs when the ray lies along (u, v) and the 
minimum when its direction is precisely the opposite. Such correspond to 
the $ asserted. The maximum [minimum] is the length [— the length] of 
the vector (u, v), which is p[—p]. 
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CHAPTER 3, 


Discrete Differential Games j 


3.1. INTRODUCTION 


Like many problems of mathematical analysis, differential games can be d 
quantized into discrete models. The smooth, continuous processes can 4 
be replaced by sequences of individual steps or moves. f 

One objective is the application of numerical methods by successive { 
calculations. With an automatic computer at hand, this may be a tempting 4 
road to the solution, especially when the analysis is difficult. But generality 4 
is lost; we cannot see how the answer depends on the starting conditions 
and on the various descriptive parameters without calculating a vast pro- } 
fusion of cases. Moreover, as some of the following examples will show, | 
many mathematical figures, such as singular surfaces or even the unique- 
ness of the optimal strategies, may be obscured or lost. { 

We do not stress this aspect here; in particular, we do not touch the $ 


question of convergence: proofs, that with a finer mesh, a discrete solution a 


approaches the continuous one. f 

Our aim is rather intellectual adumbration. At this point discrete games § 
can motivate and clarify many of the ideas to come. In the next section ` i 
we show that even the general two-player zero-sum game with full in- @ 
formation bears parallels to our theory. a 

We then go on to examples which are more literal quantifications of @ 
differential games. In Section 3.3 we present a battle game where each # 
player strives to annihilate his opponent’s forces. It might be interpreted # 
militarily or, say, as two commercial firms each striving to force his $ 
competitor out of business. We use it both to illustrate a game of degree, $ 
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ere the payoff is the force surviving on the victorious side or a game of 
d, where the objective is simply extermination. Continuous versions 
baat . : . 

such games will be easy for the reader who has mastered the ideas in 


sbsequent chapters. 
it games follow. The first, that of the hamstrung squad car, 


“Two pursul 
ctually is better adapted to the discrete rather than the continuous 
‘format. The next is the ubiquitous homicidal chauffeur game; certain of 
jts fine points come to light even here. The failure of others to do so is 
typical of the limitations of discrete methods. The final section sketches 
© <a step-by-step technique which requires only a partial quantization. 

The veteran mathematician may skip these examples without loss of 
literal instruction. They contain no material specially required for later 
use; their purpose is a preview, possible because the difficulties of analysis 
are sidestepped. On the other hand, the less mathematically sophisticated 
reader will find in this chapter the spirit of our context, through analogues 
of ideas and parallels of nomenclature, even if he reads no further. How- 
ever, the general discussion of the solutions in Section 3.2 is basic to the 
philosophy of game theory. 

Finally, let us note that it is sometimes advisable to reverse the pro- 
cedures of this chapter, that is, when confronted with a game with discrete 
moves yet with a certain logical coherence,’ we might gain by replacing it 
by a continuous model. Such is tacitly done, for instance, in the later 


Examples 5.4 and 11.9. 


32. THE GENERAL DISCRETE GAME 


Our study in this book is of two-player zero-sum games with complete 
information. Any such discrete game can be diagrammed—the so-called 
extensive form—as the sample shown in Figure 3.2.1. Rumination will 
lead to some instructive parallels with differential games. 

Each position is represented by a circle or rectangle and from each it is 
possible to go to certain others along the connecting lines in a downward 
direction. Small circles indicate that it is P’s move, that is, with the 
minimizing player rests the decision as to which subsequent position, The 
squares pertain similarly to E. Thus the topmost circle is the starting 
position; it is P’s move and he has three choices, all of which give the 
next option to E. 

We can think of a counter being placed at the starting position and the 
partie consisting of its successive moves. The occupied circle or square 
is the discrete counterpart of the state x in a differential game. Ultimately 
the counter or x reaches one of the flat rectangles which represent terminal 


1 The exact type is difficult to specify. We discuss this point later in the chapter. 
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Figure 3.2.1 






[3.2] 
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ositions; play is over and the number in the rectangle is the payoff. 
These final positions correspond to the surface @; it is clear that we have 
here a “terminal payoff.” We learned in the last chapter that any differ- 
ential game could be so formulated; we will see soon that discrete games 
can have other representations, and the present one is the particular type 
that exhibits terminal payoff. 

Chance moves can appear; then the decision as to the next move is 
made by a random device with certain given probabilities rather than by P 
or E. We have included one such position; it is the large circle labeled C, 
and the probabilities of the three possible ensuing positions are marked 
on lines to them. This random element requires us to define the Value as 
the minimax of the expected payoff. 

Let us now solve the game. From the position labeled a, E has two 
choices. Both lead to termination and the payoffs will be —2 and 0. 
His goal being the maximal payoff, he will clearly prefer the latter; let us 
Jabel this square with the number 0. From the position b, P has the same 
two choices, but hewill select —2 and this is marked in the circle. From c, 
E has three choices; the payoffs being the —2, —2, 0 as marked; the last 
is max and so square c is marked 0. In this way we work upward from the 
terminal states until finally the starting position is labeled; this label will 
be the Value of the game. The optimal strategies are represented by the 
lines that lead to positions marked with the Value. 

The chance state C is, of course, an exception. When its three ensuing 
positions have been numbered, we write in C their expectation, here a 
linear combination of these numbers with coefficients, 4, 4, 4. 


Exercise 3.2.1. Complete the solution and show that the Value of this 
game is —3/2. 

Exercise 3.2.2. Start to construct a diagram in the manner of Figure 3.2.1 
for the game of tic-tac-toe (tit-tat-toe, oughts and crosses). We say 
“start,” for the full task will be tedious and long, but we wish to emphasize 
exactly this point. 


We will now glean some insights. 

1. The numbers that were marked in each position constitute the 
analogue of the basic function in differential games which bears the 
term V(x). 

Observe that each position in the diagram can be regarded as the 
starting point of a subgame. Retaining only positions which can be 
subsequent to the given one by means of some series of moves starting 
from there, and erasing the rest, we have just the figure relevant to this 
subgame. In other words, the subgame is what follows if x were placed 
on the arbitrary given position and the players began playing from there. 
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Similarly, in differential games, V(x) is the Value of the game that 
results if a partie commences from the point x. 

2. The optimal strategies are intimately and simply related to V(x). 
When E[P] makes a decision as to his move from a position, clearly his 
best choice is that which maximizes [minimizes] V(x‘) where x‘ ranges 
over the positions immediately ensuant to x. 

A continuous version of this idea dominates what we will soon call the 
main equation in differential games. It is a first-order partial differential 
equation in V(x) and is based on the limit of the transition between 
neighboring x which renders minimax V on the later one. 

The route of x in the discrete game, when play is optimal, corresponds 
to the optimal paths in a differential game. The latter satisfy a system 
of ordinary differential equations and are characteristics of the main 
equation. 

3. Our procedure in solving the discrete game was to start at the 
terminal position and work backward, obtaining successive evaluations of 
V(x). It is hard to conceive of any alternate principle, for at the outset V 
is known only at the terminal states. 

There is a counterpart here too in differential games. We will use the 
terminal surface @ as a seat of initial conditions to integrate the afore- 
mentioned differential equations for the optimal paths in &. Such will 
motivate our reversal of time and the retrogressive form in which we will 
later obtain our integrals. 

4. Our last point is much more general and more difficult to express 
precisely. An analysis of a game such as is portrayed in Figure 3.2.1 can 
hardly be systematic; there appears no other recourse than proceeding 
one position at a time to exhaustion. The importance of this lack of 
pattern becomes manifest if we imagine the figure vastly enlarged. 

If the reader has attempted Exercise 3.2.2 he will see that, even for a 
game of childish simplicity, the diagram, even though manageable with 
sufficient diligence, is already formidable. What would it be for a game 
such as chess? Each position essentially corresponds to a permissible 
arrangement of the chess pieces on the board? and the connecting lines to 
all possible legitimate moves of the appropriate player. The magnitude of 
the diagram would be astronomical, that of the solution easily beyond 
the scope of the mightiest of present-day computers. 

On the other hand, consider the game diagrammed in Figure 3.2.2. The 
regularity is evident. A pattern of answers emerges concurrently with 
the act of solution. In fact, we can easily prove by induction on n that the 

* Certain auxiliary information is requisite too. For example, two states of a game 


with identical piece arrangement, but in only one a forfeiture of the castling privilege 
through prior moves, would have to count as two distinct positions. 
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Figure 3.2.2 






values of V in the uppermost four boxes are 


2n 2n +1 
2n—1 2n. 


Yet it is obvious that a segment of this diagram, capped at the top with 
some simple configuration that supplies the requisite opening move, would, 
if n were sufficiently large, be of greater magnitude than that of chess. 

The distinction is one of pattern—of inherent logical structure. The 
latter game yields to the techniques of difference equations or recurrence 
relations because each position is related to its neighbors with a certain 
uniformity which is lacking in games of the chess type. 

But such uniformity is the very stuff of mathematics. A planetary orbit 
is calculable in principle just because at every instant the same law of 
Newton holds. Stock market prediction is not because we cannot find a 
like logical system governing it. 

°? The reader should not be misled by the long, narrow configuration of Figure 3.2.2 
which makes it appear “one dimensional,” whereas 3.2.1 seems more “two dimensional.” 


This is beside the point; it is possible to construct “multidimensional” versions of the 
former figure relating to games similarly solvable by systematic recurrence. 


wb a 
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We hope the spirit of these ideas is clear; a sharp delineation is difficult. 
Our statement that chess does not possess the type of logical structure 
rendering it amenable to mathematical analysis does not imply that chess 
is an erratic, illogical game.* Rather it means that analysis apparently 
can but construct piecemeal chains of cause and effect; the mathematician 
appears to be able to do no more than emulate the deductions of a 
competent player. 

The quintessence of differential games is not its use of the classic tools 
of analyses, as the term “differential” might appear to imply, but its real 
concern is with games with an inner logical structure. The analytic 
techniques, such as differential equations, might be replaced by discrete 
but systematic methods once the continuous game is replaced by an 
appropriately quantitized model.’ In either case, there is a connection 
between contiguous states that maintains a uniformity of relationships 
throughout a full phase, if not all of, a partie. Such is what makes the 
theory possible. 


3.3. BATTLES OF EXTINCTION 


In the simplest form of these games there are but two state variables, 
x and y, which are the forces of the two antagonists. The game is over 
when either is reduced to zero, the payoff to the survivor being the amount 
of his own force remaining. 

For & we take the first quadrant of the x, y-plane; @ will be the union of 
the positive v- and y-axes. If y pertains to the maximizing player E, then 
the H will be —zx on the y-half axis and y on the 2-half axis. 

The options of the players, as expressed by the “kinematic equations,” 
should be, say, between moves that bring the partie to termination more 
rapidly or deplete the enemy faster. The effects should be greater with his 
own greater or the enemy’s lesser strength. 

A good simulation of any aspect of reality is hard to achieve with only 
two state variables and the above kind of moves. We do not claim any 
realistic prototype for the example below. But for a simple illustration of 
the discrete method, we must keep the dimension of & low to avoid a 
superabundance of positions. Such would be unnecessary, of course, if 
the model were solved by an automatic computer. 

In the following example we have what might well be a quantized 
version of a differential game as just described. It is not diagramed by 

* Or that it will not yield to techniques other than those of traditional mathematics 
such as self-learning computers. 

5 Such models are sometimes closer to reality, as the aforementioned Example 5.4, 


War of Attribution and Attack. In such cases, the advantage of the “differential” 
approach may be rightly viewed as one of efficacy rather than versimilitude. 


| 
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the scheme used in the last section, but its format is closer to our standard 
one of a differential game with terminal payoff. 


Example 3.3.1. A simple battle of extermination. The playing space & is 
the set of vertices of the reticulated first quadrant of Figure 3.3.1 with the 
standard x, y coordinates. Its left and lower boundary points form @ 
(where one or the other forces is zero as discussed above); the values 
of H are as marked. The vectograms are as sketched around the border. 
For example, if y = 0, 1, 2, or 3, E may make either of the two “knight’s 
moves” as shown in the lower left vectogram; fory = 4, 5, 6, or 7, he has 
the choice of the three as depicted immediately above; for y > 8, he 
employs the uppermast vectogram. 

Moves are made alternately. We think of a counter x placed on any 
lattice point (for as in the continuous cases, each point of & may serve as 
a starting position). The players move alternately by selecting one of the 
displacements of x allowed by the appropriate local vectogram. The 
partie ends when x reaches or crosses @. The payoff is the value of H at 
the point nearest to the point where the move, as represented by a straight 
arrow, crosses @. A midway crossing scores the higher in absolute value 
of the two adjacent points. 

We solve this game, as before, by finding V(x, y), the Value, if v, y is 
the starting point. But a minor difference from the general case intervenes: 










Figure 3.3.1 
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Figure 3.3.2 


either player may have first move. Thus two distinct V are required. 
In our subsequent diagrams we shall write the V, when E has the move, 
above (and to the right) of a vertex and for P, below. 

We begin by locating those points from which termination must occur 
after one move, regardless of the player’s choice. We can write in the V 
for the moving player, as it is his best possible outcome. The result is as 
in Figure 3.3.2. For example, at the point (1, —1), the two moves 
available to E lead to meeting @ at (3, 0) and crossing it at ($, 0); the two 
payoffs are 3 and 1; E picks the max = 3 and we write it over the vertex. 
On the other hand, P’s two choices lead to payoffs of —1 and —2; we 
write the latter, the min, under the vertex. We can fill in nothing yet on 
a vertex such as'(3, —2), for both of E’s moves do not reach @, but from 
(4, —2) all three possibilities do and the max is 6. 

This step is an instance of the general procedure, which is governed by 
the rule: 


When from a point x, the V relative to his opponent is known at all 
points to which a player may move, his V at x is the max (if he is E) 
or min of these V." 


Successive application of this rule determines fully the two V. For 
example, if P is to move from (5, —1), his move carries him to (3, 0), 
where V = H = 3, or to (4, —2), where V (for E} = 6; the min 3 is thus 
his V. After (5, —1) is so labeled, both outcomes of a move for E from 
(3, —2) are known and it may be treated similarly. Proceeding thus leads 
to the results in Figure 3.3.3. 


£ We will use the values of H as coordinates. 
” Of course, on &, H counts as V for both players. 
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The optimal strategies are known at once from the two V; a player 
makes the decision implied by our rule. Figure 3.3.4 depicts two instances 
of optimal play with the adjacent starting points (11, —14) and (10, —14) 
at which V (for E) = 4 and —3. The solid arrows denote the moves of E; 
the broken ones, those of P. If at a point a player has more than one 
optimal move, all are shown; thus the figures include all possible optimal 
strategies from the selected initial points. 

There is one difficulty, minor in this case, but perhaps more serious in 
possible variants, which we leave to the reader as 


Problem 3.3.1. Observe that in E’s lowermost (Figure 3.3.1) vectogram 
and P’s rightmost one, there is a pair of translations equal and opposite, 
namely, one space horizontally and two vertically. For some parts of &, 
where both these choices are admissible, a possible outcome is that each 
player perseveres in these choices so that x oscillates between two points 
and the partie never terminates. 
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Figure 3.3.4 


The rules must be amended. Let us do so simply by assigning 0 as the 
payoff of never ending play. 

Show, then, that optimal play will always end and show how to find 
the V at the critical pairs of points. 

Investigate the matter generally: find criteria for such nonterminating 
oscillations to be optimal. 


Problem 3.3.2. Note that in the upper chart of Figure 3.3.4, P often has 
choices of optimal moves, while E’s is unique. In the lower chart, on the 
other hand, E appears to have the greater freedom. 

Discuss, in a rough way, what factors govern the amount of choice 
allowed a player under an optimal strategy. 
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Suppose we were not interested in the number of survivors but simply 
wished to know which player—when striving for the contrary end, of 
course—becomes extinct first. That is, we have a game of kind in which 
the player who exterminates his opponent is winner.® 

Of course, the solution just obtained includes that of this new game; 
we have but to note the sign of V and otherwise disregard its value. But 
is there not a more direct method which does not require coverage of all 
of £? There is, and as it adumbrates the ideas of Chapters 8 and 9 on 
games of kind, we shall look into it. 

For definiteness we shall suppose E to move first. Then there is a set 
of starting points from which he can win—force x to the positive y-axis— 
and another from which he can be beaten. We should expect these two 
sets to be separated by a third set for which the outcome is a draw—x 
reaches the origin under optimal play. If we knew this third set, then, the 
problem would be solved. We can expect it to be sparser than the other 
two; in fact, it ought to be a slender array whose configuration resembles 
a curve through the origin. It is the counterpart of what later, in the 
continuous case, will be called the barrier. 


Example 3.3.2. The battle of extinction: game of kind. This game, just 
described, has the solution shown in Figure 3.3.5, where the indicated 
vertices constitute the barrier. 

To each encircled point (other than the origin) of the barrier there is 
attached one or more arrows indicating E’s optimal strategy.® The reader 
can verify that, if E makes any such move, then, whatever move P may 
make in response, x will not be brought below the barrier, nor can it 
reach the x-axis, save possibly at the origin. Suppose P brings x back to 
the barrier; E replies with another indicated move. As long as this alterna- 
tion persists, as the reader will easily see, x will be forced leftward. Should 
a move of P deliver x to a point above the barrier, then E can keep x from 
going below and compel it leftward, if anything, even the more strongly. 
Thus x must reach the y-axis; the outcome will be either a win for E or a 
draw. 

Further, the barrier is the lowermost “curve” with the above property. 
This remark illumines the construction: suppose barrier points have been 
constructed on the lines where x = 0, 1,...,”. To construct the next, 
with « = n + 1, we test the points on this line for the requisite property 
starting at the bottom and working upward, trying each eligible move of E 


8 Checkers is essentially an example of such a recreational game. 
? In Chapter 8, we shall learn that in games of kind optimal strategies are defined only 
at points of the barrier. 
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Figure 3.3.5 


in each case.” The first that passes the test is our new barrier point. The 
procedure begins, of course, with the origin. 

This construction makes clear the asserted lowermost property. In turn, 
this implies that if the starting point is below the barrier, P can force a 
win. 


Exercise 3.3.1. Construct the barrier if P has the initial move. 


3.4. TWO DISCRETE PURSUIT GAMES 


As the last example, these two will mimic the format of continuous 
differential games. They will also illustrate the compactness gained by 
using a reduced space. 

The first game, even when removed from context, appears as a neat and 
charming bagatelle. 


1° In practice, we need not be this pedantically thorough; it is clear that no barrier 
point can be lower than its left neighbor. 

11 Note that the barrier lacks the purity promised in the preliminary discussion. 
It is not comprised entirely of points for which the outcome is a draw but contains many 
which win for E. Such does not happen in the continuous case. 
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Je 3.4.1. The game of the hamstrung squad car. The squad car P 
hasing a car E of criminals through a city whose streets forma perfect, 

nbounded square lattice. Even though P has twice the speed of his 
$ arry he must obey the municipal traffic rules, which prohibit left and 
T statutes which E disregards. 

In our quantized version the players alternately perform discrete moves. 
If Eis at the vertex E of Figure 3.4.1, on his turn he may move to any of 
the four adjacent vertices of the lattice as shown. Suppose P is at the point 
Q and has come from C on his previous move. On his turn he may either 
move two spaces forward to A or execute a right turn to B. Capture 
occurs when P and E coincide or are adjacent, that is, if P is at O, E is 
captured if he is any of the nine points labeled x. Finally P has the first 
move, and the payoff is the number of moves of P until capture. ie 

Let us adopt a reduced space. We take O, P’s position, as origin of 
discrete rectangular coordinates with the vector CO of his previous move 

ointing along the y-axis. A position is then described by a point x = 
(x, y), the latter being E’s coordinates relative to P. We may think of a 
full-scale map of the city attached to the roof of the squad car, with x as 
the point of this map immediately over E. 

The disadvantage of a reduced space is that moves generally become 
more complicated.!2 Suppose that P moves two spaces forward to d; 
such is equivalent to moving x (as on the figure) back two spaces to 4’. 
Now let P turn right to B. Orienting ourselves along the vector OB, we 










































































Figure 3.4.1 


12 In the continuous case, it is the KE which suffer similarly. 
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Figure 3.4.2 


see that the x illustrated is four spaces to the left and one forward of B, $ 
Thus the effect of the move is to transfer x to B’, where these coordinates 3$ 


hold in the original scheme. 


Now for the solution! Again we achieve it by successively calculating j 
V(x) from the capture region outward. We will label the vertices with a 


their values of V as we go. 


We begin by marking the origin and its eight neighbors 0, for at these 
points P captures with 0 moves. Inasmuch as he moves first, we can | 
immediately spot the points where V = 1; they are marked on Figure 


3.4.2. 


From here on, the process assumes generality. The next step is to 
delineate the points hemmed in, from E’s viewpoint, by 1’s or 0’s, that is, | 
the points x such that if E were at x all four of the points to which he : 


could move have already been marked, at least one of them with a 1. 
These points are enclosed by the broken line in the figure. Next we spot 
the points not previously marked such that a move by P will bring them 


to one of the enclosed positions. There are two such points (a) if P moves ` 


forward and two (b) if P turns right. The four points, a and b, are labeled 2. 
Let us verify the last step. If x is at one of the points a[b], P can bring x 
to the enclosed set by moving straight [turning]. (Such moves are part of 
his optimal strategy.) It is now E’s turn; his optimal strategy demands 
that he shift x to a point with the highest possible label, which here is 1.8 
It is now P’s move, and we already know (note the induction!) that he 
can capture x on such a position in one move, making a total of two. 


1° Tt is even better for E to move to an unlabeled point if he can. Our construction 
makes this impossible here. 





: a four neighbors of x are marked, with at least one mark an n. The 
points, if any, not already marked and such that one move of P brings 
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_ The ensuing steps are similar. Suppose the points with V = 0,1,...,m 
, ave all been found and marked. Let S consist of the points x such that 


them to S, are now marked n + 1. ' i FN l 
The reader who executes this construction for himself will enjoy a mild 
and diverting task. When V = 11, he will have constructed Figure 3.4.3. 
He will find that no further steps are possible; the configuration is com- 
lete. , 
If the game is started from an unmarked point, E can permanently 
evade capture. (What is his strategy ?) 


Research Problem 3.4.1. If we increase P’s “speed” by letting him move 3 

or more spaces on a move instead of 2, is it still true that there are starting 
oints of permanent escape for E? The capture region should similarly be 

enlarged to eschew the possibility that P pass over E without capturing 

him. 

Exercise 3.4.1. Chart some trajectories of optimal play in the realistic 

space. 


Example 3.4.2. The homicidal chauffeur game: a discrete version. Our 
discrete version of this game will be played on a triangular lattice, but in 
most other respects the ideas of the last example persist. Again moves are 
made alternately, with P going first; he moves two spaces at a turn, 
while E moves one. 


8 u 
5 8 
3 4 7 10 
2 347 
1 1136 9 
i l 1236 
A 
0001158 
o Qos 1 2 3 
9000113 4 5 8 
7432384781 
10 7 4 3 6 7 10 
8 5 6 9 
11 8 


Figure 3.4.3 
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[3.4]! 


If P is at O in Figure 3.4.4 and has come from C on his previous move,’ 
then he must choose between the three moves which take him to A, B, or 
D. In this way we have a simulation of a fast pursuer subject to a limited: 
curvature of path. On the other hand, E may move one space to any of: 
the six adjacent points as shown. 

Capture occurs, if P is at O, when E occupies O or any of the six con. 3 
tiguous points, all labeled x in the figure. Again the payoff is the numberd 
of moves of P until capture. 

Once more we employ a reduced space in which P is at the origin 0,] 
his prior vector CO being on the vertical axis, and x is E’s affix in this 
coordinate system. We leave the reader to apprehend the displacements] 
of x in response to the three possible moves of P. ; 

The technique of solution is the same as that of the previous example:: 
we determine V starting from the capture region (the x) and working 
outward. The points where V = 0 or 1 can be located immediately. As 
before, when we know the points where V = 0,..., n, we first locate the§ 
set S of x such that if V is at x, for all six contiguous points V < n and! 
max V =n. Then the set where V = n + 1 are those points to which no V 
has been previously assigned and, for some one move of P, can be brought} 
to S. 3 
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Figure 3.4.5 


What results is shown in Figure 3.4.5. This time all lattice points are 
marked; P can always capture. Because of symmetry, one side is omitted. 
Observe the successive values of V. Near the top of the chart they occur 
consecutively in orderly concentric rows. The ends of each row curl 
around that of its predecessor, and this happens more and more until 
V = 9. The row of 9’s extends right across the bottom of the chart leaving 
a cavity near the tail of the double arrow, which is filled by V’s ranging 
from 10 to 13. These entries consist of the starting points that lead to 
optimal play with a swerve (Section 1.5). 

An instance in the realistic space, starting from the encircled point 
where V = 12, is plotted in Figure 3.4.6. Note how P must go twice 
leftward before veering back to get at E; note how E at first follows in an 
effort to intensify P’s need for this maneuver. 

This is not the only outcome, for in this game the optimal strategies are 


a 
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often not unique. The cause is merely the quantization and is easy ¢ 
apprehend. Suppose, say, E wished to travel a long stretch horizontal} 
(in terms of the orientation of the figures) as rapidly as possible. He woul 
have to zig-zag and at alternate points would have two equally good mov 
(he could go slantwise up, then down, or down-up). 

For reasons such as the above many of thi 
niceties of the homicidal chauffeur game bg 
come blurred in its discrete version. In lat 

chapters we shall study a curve called th 
barrier which delineates the starting poing 
leading to a swerve. On the arrier V is dig 
continuous. The counterpart can be crudeff 


= seen in Figure 3.4.5 by looking for pairs & 
7 adjacent points at which V differs by mog 
Da than 1. i 


i 8 3.5. QUASI-DISCRETE GAMES 
An occasional use of discrete models is tha 

Ny sometimes, when we are perplexed about 
E differential game or some phenomenon therë 

| in, we can gain a foretaste of the truth by g 
step-by-step solution of a discrete analogué 
Sometimes better for this purpose are model 

|e that are partially discrete, partially contin 
uous. We will sketch lightly one such possig 

bility. 3 

Let us retain the continuous vectograms of 
11 a differential game but decompose time intq 
a series of fixed (and usually equal) intervals 


The players move alternately. A move lastg 
| ae an interval and during it the control variabkg l 


Pa are kept fixed. 


Figure 3.4.6 We will again use the homicidal chauffe $ 


game. Following the pattern of the la 


example, we take vectograms?* as in (a) of Figure 3.5.1. On a move ağ 
player makes a displacement of a prescribed magnitude, but he has ag 


choice over a continuum of directions. 


The stepwise construction of V begins as shown at (b) of the figure. 


14 We are not following exactly the dictates of the previous paragraph i in that wea 


not basing our work on the ¢- vectograms of the continuous version in the reduc d : 


space. Such vectograms will be described in Section 10.2. 











QUASI-DISCRETE GAMES 63 


(a) 





(b) 
Figure 3.5.1 


Using the same reduced space as before, the capture region will be, say, 


the circle @. The first step to find the subset S, of @ of points x such that 
one move by P will bring x to or within @. The set S, will be bounded 
underneath by part of @ and above by curve such as A, in the figure; it is 
just the set where V = 1. Next we find the subset of S, which inhibits 
escape by E, that is, the points x of S, such that no move by E can cause x 
to leave S}. In the sketch it lies below the curve B. The next cycle starts 
by finding Sz, the set of points not in @ or S, such that some move by P 
will bring them to the above set between B and @. Then S, is the set 
where V = 2. We continue so. 

An actual execution of this procedure would seem to be extremely un- 
wieldy if n exceeded 2. For a planar case as described, a practical method 
is to use two sheets of tracing paper, one for the A curves, one for the B, 
and alternately trace from one to the other. 



































CHAPTER 


The Basic Mathematics and thy 
Solution Technique in the Sma 


The fundamental mathematical concepts that will permeate the rest gj 
this book appear here.1 The main equation, a first-order partial differentig 
equation for the Value, V(x), is derived by two methods. Its appropriaf 
integrals can be shown actually to be the Value of the game in eac 
particular case by what is termed the verification theorem. The theorem é 
proved and its use illustrated by a varied sequence of examples. 

The final sections construct our standard method for obtaining solution 
in the small, that is, in all cases where there is no interference from singula 
manifolds. 4 


4.1. THE NATURE OF A SOLUTION 


When a particular differential game has been solved, the results wil 
generally embody such entities as 


1. The Value: the function V(x) defined over &. | 


2. The optimal strategies: (vector} functions (x) and Ẹ(x) defined ove 


&. They may or may not be unique. In the latter event we might beg 


interested in obtaining the complete set of all optimal strategies or wél 


might be satisfied with just one. = 
3. The optimal paths: when either the optimal strategies are unique ong 


for some reason, we have decided to devote our attention to a uniqué 


choice, there will be a set of optimal paths which are traversed by x whet 
these strategies are played. These paths should fill & and each shouldg 


terminate on @. 
1 A re-edited version of RAND Report RM-1411 (21 December 1955). 
64 





THE NATURE OF A SOLUTION 65 


rhere will be cases where these entities do not all exist, at least not in 
‘of & The later Example 7.8.1 is an elementary instance. There are 
ye subtle types; some counterexamples in works on the calculus of 
jations would be such instances of one-player games. In such cases we 
d a solution as the supplying of as much enlightening information as 
he circumstances warrant. 
We prefer not to hedge in the concept of a solution by too rigorous a 
finition. In the event of the nonexistence, in all or part of &, of the 
tities 1, 2, 3 or some other pathological aspect, we will regard the game 
; solved when these phenomena are elucidated and understood. Non- 
tence is not a calamity; usually there is a simple and enlightening 
planation. 
‘Even in completely nonpathological case, when 1, 2, 3 all exist without 
ymplications, it is not always necessary to specify explicitly all three. 
or example, if V(x) is known, the optimal strategies, as we shall shortly 
, can be computed as known functions of the state variables and the 
partial derivatives of V. If the optimal strategies are known, the optimal 
paths follow by integration of the kinematic equations; if V is integral, it 
becomes known through a functional elimination. 

There will be cases where processes of the latter types are perfectly 
routine in principle but oppressively tedious to compute explicitly. Whether 


¿itis worth the trouble of doing so depends on what motivated the problem. 
If it was to illustrate ideas, as are many in this book, then often the fruits 


of the labor bring no further lucidity. Even with a practical problem it may 
be only certain aspects of the solution that will find application. 

Thus, in the ensuing examples, in this regard there will be considerable 
variety, guided by judging the value in interest against the cost in labor, as 
to how detailed are the solutions presented. 

To be purely logical, the solution should be expressed in terms of 

K-strategies. However, we will reserve this concept as an ancillary tool 
with which to give solutions a rigorous meaning when this is deemed 
necessary and to prove their validity. We shall demonstrate how such is 
done later in this chapter. In the future we will take the possibility for 
granted; we shall speak of strategies rather than tactics and generally 
think in terms of integrations of the kinematic equations. 
; We recall that the data of a game include a particular starting point 
in @ and that we have used the term “game” rather freely for what 
should be a family of games. When we speak of the solution of a game 
for a certain subset &” of &, we will refer to all games with starting 
points in £”. 

The process of solving a game splits into two phases. It generally turns 
out that the region £ is to be divided into a number of parts separated by 
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the surfaces which we shall later call singular surfaces.2 In each part the $ 


solution will be smooth, that is, V will be of the class C}. By this we mean 
that surfaces of constant V have continuously varying tangent planes; 
analytically, for all small vectors u = (t4, ..., Un) 

Væ + w) — V(x) = $O a, + Jul oup. (4.1.41) 

Similarly, except possibly on, singular surfaces, the optimal strategies 
will be continuous functions denoted by (x) and #(x) when they are 
unique; when they are not, we shall often assume such continuous func- 
tions can be selected. 

On the singular surfaces a variety of special kinds of behavior can occur. 
This diversity has thus far defied a systematic theory which can be special- 
ized to treat each type. A classification and notational scheme will be 
attempted in Section 6.1, but it does little more than catalogue the 
possibilities. Much of the space devoted to theory in this book concerns 
singular surfaces, and the theory of each type is distinct. 

We shall at times use the term in the small to refer to the smooth parts of 
the solution found between the singular surfaces. The problem of identify- 
ing the singular surfaces and assemblying the smooth parts into the total 
solution will be described by the phrase in the large. 

We shall see that the technique in the small is one of differential equa- 
tions. This phase of the problem plays a part in full solution somewhat 
analogous to that played by the Euler equations in the calculus of varia- 
tions. But we have been impelled to innovations of technique. Although 
our methods can handle the classical problems by viewing them as one- 
player games? (the second player is inactive in that his vectograms are null), 
we cannot do so through “extremals” but must distinguish minimum and 
maximum at the outset; how else could we handle games? 

Not all types of singular surfaces are foreign to the classical calculus of 
variations.* But until we adopted the present game theoretical point of 
view, there was no compelling motive for identifying them, and their 
presence has been tacit. In our work, not only is the variety richer but the 
type of problem that game theory makes prominent emphasizes their 
importance. 

It is hard to make a categorical statement as to the relative importance 
of the in the smail or in the large phases. In some problems the integral 


* There may also be singular manifolds of dimension <» — 1; such of course cannot 
separate the components of &. In the theory as thus far developed, they have been 
rather neglected. The term surface means an (n — 1)-manifold in n-space. 

3 See, for example, the dolichobrachistochrone problem (Section 5.2). 

4 See, for instance, Example 7.2.1. 
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solutions between the singular surfaces are simple, but the latter them- 
selves are numerous, varied, and difficult. In others the integrals yield a 
rich family of paths which fill € with little or no singular behavior. The 
homicidal chauffeur game is an instance of the former; in the next chapter 
(Section 5.5) the game of the isotropic rocket, a variant of the same 

roblem, falls in the latter category. The nearest we can come to a general 
criterion is to say that linear vectograms (the control variables appear 
lineally in the KE and G) imply many singular surfaces; indeed certain 
types can occur only in such linear cases. 


4.2. THE MAIN EQUATION 


We suppose that the Value of a differential game exists. It will depend 
on the starting point x and we denote it by V(x). We shall show that V(x) 
satisfies a first-order partial differential equation, to be called the main 
equation (abbreviated ME), whenever V(x) is of class C,. We shall write 
here and in the future 

ov ; 
ar J Careers 
The ME is 


min max > [vif $, y) + G(x, 4, »| = 0. (4.2.1) 
¢ yp j 


Summations without limits, as written here, will be understood to run 
from 1 to n, where n is the dimension number of &. 

From the minimax assumption, it makes no difference if the min and 
max in (4.2.1) are reversed. It is understood that they range over all 
(vectorial) ¢ and y which satisfy the constraints. 

We shall give two proofs. The first will follow immediately. Our 
treatment ‘will be heuristic, inasmuch as we shall erect our serious founda- 
tions on the second proof, but the mathematician will easily see how to 
render it rigorous. The advantage of this first derivation is its straight- 
forward, instructive character. It is the continuous counterpart of the 
process used for solving the discrete games of the preceding chapter. 

We utilize what might be called the tenet of transition. The germ of the 
idea is that we are dealing with a family of games based on different 
starting points. Let us consider an interlude of time in midplay. At its 
commencement the path has reached some definite point of & We 
consider all possible x which may be reached at the end of the interlude 
for all possible choices of the control variables by both players. We sup- 
pose that, for each endpoint, the game beginning there has already been 
solved; in other words, V is known there. Then the payoff resulting from 
each choice, $, y during the interlude will be known, and the control 


a 
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variables are to be so chosen as to render it minimax. When we let the 
duration of the interlude approach zero, the result yields a differential 


equation. 
We put the above reasoning formally. Let V be known at x in & which 


has been reached in a partie at time ż. A short time later—t + h—the 
play has progressed to the (variable) point x°. Then 


tth 
P(x) = payoff at x -Í G(x, $, y) dt + V(x°) 
t 


for during the interlude (¢, t + h), the payoff acquires an increment equal 
to the above integral which, to get A(x), must be added to the payoff at x° 
(which, of course, is the sum of such an integral extended over the remain- 
ing time of play and H at the terminal point). We assume that play from x° 
is optimal, so that the payoff here is V(x°). The idea of our reasoning is to 
regard x as fixed but to admit various x°, which arise from all possible 
choices of $ and y during (t, t + A). 
We shall replace the integral by a Taylor expansion of the type 


[To dx =hf(t)+ 4h?f'(t + 0h) 0<0<1. 


In the final term we shall write 
xX=x+u 


where, very closely for small enough h (because f; is ż; for the ¢ and y 
chosen), 

u; = f(x, $, yh 
and we shall use (4.1.1) to make a further replacement. When all these 
have been done, we find that 


Pex) = V + hl Glx, HW) + E Vif b, v) + a(t) 


where the ¢ and y stand for their values at x5 and « goes to zero with A. 

We are to take the minimax of F with respect to ¢ and y. Such means 
applying this operation to the bracket. But, as by definition the minimax 
of F is V(x), the minimax of the bracket equals zero. Letting h— 0, 
we have (4.2.1), the ME. 

We shall refer hereafter to (4.2.1) as the first form of the main equation 
and designate it by ME. If we actually ascertain the minimax, the ¢ 
and y (or a judicious choice if there is more than one possibility) that 
supply it will in general depend on x (= 2,,...,%,) and the V, (i= 1,...,m). 


5 Or better, some mean value during the interlude (z, £ + A). 
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It will be convenient to write the latter vector {V, Via 
4 s pe ees s V. Ki 
the minimax in the ME, will be furnished by : a} as Va® Then 


$(x, Va) and G(x, V,). (4.2.2) 


When these functions are substituted into the ME,, the bracket will be 
zero; we will have 


F Vifilx, $x, Va), Pe, Va] + GTX, $x, Ve), Hx, Ve} =0 (4.23) 


a first-order partial differential equation in V which the Value must satisf 
We will refer to (4.2.3) as the second form of the main equation and cae 
it by ME,. 

We have already used 4(x) and #(x) to denote optimal strategies. Note 
that (4.2.2), as they stand, are not Strategies at all, for we are presuming 
that at this stage we do not know the V, However, in practice, this 
conflict of notation does not appear to create any confusion and seems 
preferable to some novel symbol, such as $ , for (4.2.2). Note that once V 
is known and its partial derivatives inserted for the V, in (4.2.2) these 
functions become the optimal strategies d(x) and Ẹ(x), a fact which 
ameliorates the above possible confusion. 

In the future, when an ME, is written, to save space the arguments x, V, 
of $ and % will often be omitted. We shall take pains to label the equation 
ME,, so that the reader may be aware of these unwritten symbols, 


Exercise ike . The following KE will be those of Examples 4.4.1 to 4.4.5 
to come (however, their comprehension is not necessary for thi ise 
The payoff is terminal and the KE are á oper 
#=up+wsing 
ğ = —1 + woos ¢, -l<y<l 


where u and w are smooth, positive function i 
s of and y. Write t 
and show that the ME, is ne 


uV — wp — V, =0 
where, if p = y V2 + VR, 
P = sgn V, 
sing = —V,/p, cos¢ = —V,/p. 
(Use Lemma 2.8.1.) 
ê The customary notation being grad V. This notation is generally not required in 


particular problems. i . ; 
7 problems. In such, if x denotes a state variable, V, will of course mean 
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Exercise 4.2.2. The homicidal chauffeur game. Write the ME, and ME, 
for both the KE (Example 2.1.2) in the realistic space and the KE (Example 
2.2.2) in the reduced space. One notation for the former will be V,,..., Vg, 
the subscripts pertaining to the state variables in the order in which they 
appear in the KE. (The reader will find the required ME in subsequent 
pages.) 

Observe that along an optimal path 


V= > Vf, $, p) = —G(x, $, p) 


and so V is constant on all such paths if and only if the payoff is terminal. 
The second proof of the main equation depends on a new concept: 


43. SEMIPERMEABLE SURFACES AND A SECOND 
DERIVATION OF THE MAIN EQUATION 


We take it that each small portion of the surfaces under discussion 
separates the neighboring space. As orientation is germane to our purpose, 
we distinguish the two directions in which the surface may be penetrated, 
calling them the P- and E-directions. The “side” of the surface reached 
after penetration in the P-[E-] direction will be called the P-[E-] side. We 
take a point x on a so oriented surface and visualize the full vectogram at 
x. We will say the surface is semipermeable at x when the following is true: 

There is at least one value $ of ¢ such that if $ = 4,” no vector in 
y-vectogram penetrates the surface in the E-direction. Similarly, there is 
a p” which prevents penetration in the P-direction. 

A surface having this property at each point will be called a semi- 
permeable surface, abbreviated SPS. 

We have already seen (Theorem 2.4.1) that we can transform any game 
into one with terminal payoff. Consider an instance of the latter which we 
suppose solved and for which V(x) has at least two values. 

Any surface which separates the parts of & where V >c and V <c 
(c, any constant) must be semipermeable with V decreasing as the surface 
is crossed in the P-direction. For if, at some point x of the surface, there 
were no ¢, P could not prevent E from pulling x into the side with the 
larger V. Similarly, there is a #. Thus, for x on the surface, the players 
must employ ¢ and %, and these are the optimal strategies there. 


7 $ip] will be used interchangeably to denote a value of ¢[y] with the described 
property or the set of all such values. Even though we seem to have burdened the 
barred control variable with still a third meaning, it will soon coalesce with our former 
usages. 
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Now suppose in a certain subregion of &, V is of class C, and constant 
jn no neighborhood. Then the surfaces on which V is constant will be 
semipermeable. The vector V, = {V;}is normal to such surfaces. Whether 
a moving point penetrates the surface in one direction or the other or not 
at all depends on the sign of its velocity component along this vector. 
That is, the semipermeability condition for the surfaces of constant V is 


min max > V,f,(x, $, y) = 0.8 (4.3.1) 
p t 


But this is the main equation in the terminal payoff case. 

If we regard V, as the flow velocity of some substance in £, (4.3.1) can 
be interpreted as implying no flow across a SPS when ¢ and # (yielding the 
min and max) are employed. Using but one of @ and ¢ and leaving the 
other free, we see that each player can prevent a transverse flow across 
the surface in his direction. Hence the name, semipermeable surface. 

Suppose we had begun with a general game and used the transfor- 
mation of Theorem 2.4.1 to obtain one with a terminal payoff. If there 
were n state variables originally, the sum in (4.3.1) will have n + 1 terms, 
and f,1 Will be G. We know that in the transformed game, H is (the old 
H) + Sn, and the new optimal paths are translations of the old in the 
41 direction. Thus if, at any starting point, x,,, is increased, V will be 
increased by the same amount. Therefore V,,,, = 1. 

The preceding two replacements show that the final term in the sum of 
(4.3.1) is G and so this equation is identical with the ME,, (4.2.1). 


The preceding conception furnishes an outlook on differential games 
with rather forceful intuitive appeal. For simplicity, let us envisage a case 
where everything turns out to be smooth; there are no singular surfaces.® 
The payoff is terminal; we know this can be made true of any game. 

On @, the function His given. We assume the curves [(n — 2)-manifolds] 
on of constant H cover this surface in a regular way. Now, suppose we 
succeed in filling & by a family of surfaces with just one passing through 
point of &, such that 


1. They are SPS, suitably oriented. 
2. Each meets @ in a curve of constant H. 


(Our technique of solution, a basic constituent of this book, is logically 
equivalent to the construction of such a family.) 


* A more detailed interpretation of this equation’s being characteristic of smooth 
SPS will be found in Section 8.3. 
° Such is generally true of the in the small constituents of more general cases. 
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Then it certainly seems reasonable to assert that these surfaces are of 
constant V (the constant being the value of Hat @). For at any point of &, 
the minimizing player P, when suitably opposed, cannot cause x to 
penetrate the local surface to one of lower V. Nor can E, similarly, 
attain a higher V. In fact, to prevent his opponent from doing better than 
V (which labels the local surface) each is compelled to use the non- 
penetrating $ and # which appeared in the definition of a SPS. As long as 
they continue to do so, x remains on the same surface; a defection by 
either player makes it possible for his opponent to penetrate to a more 
advantageous one. 

Now, if continued play with ¢ and # will bring x to G, we feel a strong 
assurance that the labels on the surfaces are values of the Value. The 
italicized assumption above is typical of the domain of games of kind. 
We will broach the subject in this chapter with two typical illustrations 
(Examples 4.4.4 and 4.4.5). 

The theorem of the following section is an adaptation of the foregoing 
ideas to K-strategies. 


4.4. THE VERIFICATION THEOREM 


Inasmuch as the diversity of phenomena that will soon be seen to arise 
in even the most typical problems is great enough to preclude an adequate 
existence theorem, we shall adopt another approach. We shall develop a 
technique for solving problems. The question will then be whether the 
formal solution so obtained—the Value V(x) and the accompanying 
optimal strategies (x) and #(x)—really solves the problem, and if so, in 
what sense. 

The sense will be that of K-strategies. The means is Theorem 4.4.1. It 
may be applied separately to the stages of the solution process, each of 
which is essentially the finding of an integral of the main equation which 
satisfies the proper boundary conditions, such as agreeing with H on @. 

We shall state and prove Theorem 4.4.1 and demonstrate its use with 
some examples. Let us recall our general assumptions that each /,(x, $, y) 
is a continuous function of its three arguments and the constraints limit $ 
and y to sets; we shall call these sets E; and E,,, and they are compact and 
independent of x. First the 


LEMMA 4.4.1. If and y are held constant and x is confined to a compact 
set E,, then f(x, $, y) is uniformly continuous with respect to x and all 
possible constant values of ¢ and y. 

Proof. We have to show that for each e > 0, there is a ô = d(e) such 
that 

Ifo y) — f 0 p, vl < e 
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for all x}, x? € E, with 
[xt — x? < ô 


and all dc E, and we E,. But f is a continuous function of all three 
(vector) arguments in the compact product set E, x E, X E, and hence 
uniformly continuous therein. The lemma is a special case of this uniform 
continuity. 


THEOREM 4.4.1. Let &” be a subregion of & neighboring a subregion @’ 
of @ in a game of degree with terminal payoff. If a function V(x), defined 
in £’, enjoys the properties: 


1. It satisfies the ME, (4.3.1) in & 

2. It is of class @, in &’ 

3. It equals H on @’ 

4, It is the only function satisfying 1, 2, and 3. 

If $(x) and #(x) are any functions which furnish the minimax in (4.3.1), 
then V(x) is the Value in &” in the sense of K-strategies and $ and @ are 
optimal tactics, provided that x reaches @’ from any starting point in &’. 

Proof. In Section 2.7 (end), we learned that, by a change in the time 
scale, we can bound all speeds in each full vectogram in & without 
changing the game in any essential way. Let us do so with the bound < 1. 

Let us select a tactic $(x) for P such that for each x, $ is minimizing in 
(4.3.1). Let E play any K-strategy; let y(x) be its tactic. Play starts from 
x’. Givenan £ > 0, we are going to complete P’s K-strategy by constructing 
a o, We shall speak as if P were to play indefinitely; of course, we need 
but curtail our scheme when @ is reached. 

First we divide time into unit intervals J,: n < t<n+1(n=0, 1,2, 
...). During I, x cannot have traveled more than n + 1 from the starting 
position x° and so is constrained to a compact set. As each V, is continuous 
by 2 and from the lemma, the function 2,V,(x)f,(x, $, y) is uniformly 
continuous in this compact set when ¢ and y are constants and uniformly 
with regard to these constants too. 

Pick m, so that if x changes in distance by less than 1/m,,, the quantity 
2,V;f; will change by less than ¢/2"+1, Next subdivide J, into m, equal 
parts, completing o,. Then, during the passage of any of these subinter- 
ludes, the change in %,V,f, suffers the same bound as above. A sub- 
interlude may again be subdivided by one or more t, from E’s K-strategy. 
But at the beginning of each of length 1/m,, because P then used a 
minimizing ¢ in (4.3.1), we have 


2 ViLAX, $, y) < 0. 


The arguments 4, y being the piecewise constant tactical values from the 



































74 THE BASIC MATHEMATICS [4.4] 


K-strategies played, the left side cannot rise to ¢/2"*1 during any such 
interval and therefore cannot throughout 7,. 

Now let us consider the path of x under the preceding K-strategies. 
In each of the ultimate subinterludes, when both ¢ and y are constant, it is 
an integral of the KE with these constant arguments; thus the path is a 
“polygonal” sequence of smooth segments. On the path as a whole V is 
continuous, and on each segment dV jdt exists and is given by 2, V,f,(x, $, y) 
with appropriately constant ¢ and y. From the preceding paragraphs it 
follows that the growth of V is less than £/2”+! during J,, and hence always 
less than e446 + 4+...) = 6. 

By definition and from 3, the payoff will be the V(x) when x reaches 


@'. Therefore payoff < V(x) + e. 


Similarly, we can construct a o; for E ensuring a payoff >V(x°) — e. 
Thus V is the value. 


The application of the verification theorem may be multiple; such is 
accomplished according to the following general idea: Let £’ and @’ be as in 
the hypothesis of the theorem. Pass a second surface @” through @’. If 
we use this surface as we did @’, taking for H on @” the values of V(x) 
obtained from the first solution, the new solution on the side of @” away 
from @’, will agree with the old. This principle permits the validation of a 
complex solution in consecutive stages. Singular surfaces of various 
types can be handled as @” above, referring now to a new phase of the 
solution rather than confirming an old one. 

What of the very embracing final clause of the theorem? Whether or 
not termination will occur is a matter of games of kind and their treatment 
too falls under the aegis of K-strategies. The subject is treated in detail in 
Chapter 8, but we must make some anticipations here. 

Since we are dealing with games, we shall suppose one player, say P, 
desires termination while his opponent does not. In the most interesting 
cases (and the only ones discussed in this book) both termination and 
nontermination starting points exist. The former are such that P has a 
strategy ensuring termination against any opposition; the latter are such 
that E can similarly prevent it. The two sets of points are separated by 
a surface, which must be semipermeable, called the barrier.1° Let us 
imagine the barrier imbedded in a lamina consisting of neighboring 
“parallel” SPS and define (temporarily) a smooth function U(x) on the 
lamina which equals zero on the barrier, is strictly decreasing in the 
termination or P-direction, and constant on each SPS. Then U satisfies 
the same equation (4.3.1) as V and so we can construct a K-strategy for P 


10 These ideas are amplified in Chapter 8. 
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as in the preceding proof. For any starting point x° on the termination 
side, so that U(x°) < 0, if we use an £ < —4U(x), then for all x of the 
ensuing path, U(x) <0. Then, no matter what K-strategy E plays, he 
cannot make x cross the barrier into the nontermination points. Similarly. 
for starting points on the other side, E will have a K-strategy deterring 
entry into the termination region. 

Once the question of termination or not is settled in this or a similar 
way, we can apply Theorem 4.4.1 with & consisting only of the termination 
points. We now have the requisite assurance that termination will occur. 

Throughout this book we shall solve many examples in the sense of 
ascertaining entities V(x), $(x), @(x) as discussed in Section 4.1. In principle 
we should apply the verification theorem to each to prove that the formal 
solutions are actually such. But as this would lead to a pattern of monoto- 
nous replication, we shall give here a series of simple yet typical illus- 
trations—all variations of the same example—which will clarify many 
aspects of the general procedure. 

This chain of examples will unavoidably necessitate our anticipation of 
certain later ideas. The reader can either read these cases lightly now and 
return to them after subsequent ingestion, or he can do some (light) 
reading in the requisite advanced pages. 

We pause first to recall the germ of the K-strategy idea. When we have 
found, say, $(x) and employ it as an optimal tactic, it can be pitted 
against any (x) which fulfills the constraints. We are freed from putting 
any constrictions of function type, such as piecewise continuity, differentia- 
bility, etc.,,on the opponent’s strategy. 


Example 4.4.1. & is the upper half-plane and @ is the x-axis. The vecto- 
gram for E is shown at Figure 4.4.1a; it has a downward unit vertical 
component and a horizontal headline? of half-length u(x, y), a positive and 
smooth function. That of P is circular of radius w(x, y), again a smooth, 
positive function with always w < u and, for some constant c, w < c < 1. 
The total velocity for æ is to be the vector sum of a choice from each 
vectogram. Analytically all this means that the kinematic equations are 


& = u(x, yy + w(x, y) sin d 


ġ = —1 + wz, y)cosġ, —-lgy<l. 


The payoff will be terminal with H = x on @ (where y = 0). Thus E 
will strive to have x reach @ at a point as far right as possible, and P 
similarly struggles for the left. 

Always E will play his rightmost vector (y = 1 in the KE). At (b) of the 


11 That is, the line of “arrowheads” of the vectors. 
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@ 
(c) 


Figure 4.4.1 


figure let XA be this vector. The dashed line XB is tangent from X to a 
circle of radius w (w is reckoned at X)and center A. Then XB is a properly 
oriented semipermeable direction. If a family of curves is drawn (an 
ordinary differential equation solved) having these directions as that of 
their tangents at each point, these curves will be SPS. If each is labeled 
with the value of H at its meeting with @, the labelings will constitute V(x). 

These assertions either follow from our analytic methods still to come or 
they can be attained geometrically. To see that XB is semipermeable, 
consult Lemma 10.2.2, which deals with a very similar situation. 

The curves of constant V, which are also the optimal paths, perhaps 
appear as in (c) of the figure. The four hypotheses of Theorem 4.4.1 are all 
satisfied by the above V(x). Further, as the downward velocity of x is 
always at least l-c, we are assured that every partie will terminate. Thus 
there is no difficulty in asserting the theorem’s conclusion. 
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@ 
Figure 4.4.2 


Example 4.4.2. The same, except we change H. It is increasing except on 
a certain interval where it is constant, say d; its graph is as in Figure 
4.4.2a. Such causes a region (shaded in (b) of the figure) in & where V is 
constant; in fact V = d. 

Because here all V, = 0, the main equation is certainly satisfied. As any 
¢ and w give (trivially) the minimax, all are optimal. Considering the 
nature of the game, this is certainly true in the shaded region. The 
theorem holds. 

The point here is simply that, whereas V is unique, 6 and # do not have 
to be so. 


Example 4.4.3. Again the same, except that now H = 2%. Thus E will 
strive to have x cross @ far from, and P near to, the point where x = 0. 

It is evident that at points far to the right in &, our previous construction 
holds. For points far to the left, a symmetric one holds with E playing his 
leftmost vector, etc. 

Thus we construct two families of paths as shown in Figure 4.4.3, 
Chapter 6 deals with such cases; there we learn that we should delete the 
paths after they cross (in the sense of traversing them from @) a curve D 
which is the point set where the two functions V(x) are equal (2 will later 
be termed a dispersal surface). 

For x on either side of 2 we apply the theorem, but we must take & as 
that part of & lying on one side of 2 and @’ the part of @ lying on the same 
side of O and 2 itself. For either part H is taken as given on the retained 
part of @ and as the mutual value of V on 2. In this way all hypotheses of 
the theorem are fulfilled and we can appropriate its conclusion. Observe 
that no other dividing curve but 2 can accomplish this result. 
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Figure 4.4.3 


For starting points on 2 each player is faced with two choices of values 
of his control variables. Each should play a mixed strategy making his 
choice with probabilities (4,4). This matter is discussed at length in 
Chapter 6. 

There are two points to note here. We curtailed part of the solution of 
the main equation: therefore, not all of every formal solution V(x) need 
be the Value of the game. A more subtle instance occurs in connection 
with equivocal surfaces and will be expounded in detail in Chapter 10. 

The second point is that 2 is a singular surface; although V(x) is 
continuous, certainly all its first partials do not exist there. The theorem 
is quite capable of dealing with problems that have singular surfaces when 
judiciously applied. 


Example 4.4.4 We now let & be the entire plane and @ the positive 
x-axis. H can be taken as x as in Example 4.4.1. 

What is new is that from starting points far to the left x may miss @ and 
there will be no termination. It is more interesting to have P desire 
termination and E, its avoidance. That is, E wishes to escape (nonter- 
mination) if possible; if not, he wishes x to meet @ as far to the right as he 
can occasion. 

We could realize the foregoing situation numerically by assigning the 
payoff + œ in the event of escape, but such seems a rather empty formalism. 

We have explained very briefly, on page 74 the nature of a barrier and 
referred the reader to Chapter 8 for fuller details. In our case the barrier 
will be the “left-hand” semipermeable surface, in the sense of Example 
4.4.3, which passes through O. It is marked # in Figure 4.4.4. 

As was explained earlier, if x lies to the left of Z, E has a K-strategy 
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which guarantees x’s not crossing # and hence ensures escape. Similarly, 
if x lies to the right of Z, P has a K-strategy which deters crossing (in the 
opposite sense) and so ensures termination. (We make no statement now 
concerning the case of x’s starting from 2.) 

For starting points to the right of @ and above @, both players, knowing 
that capture is inevitable, play the game of degree. The reasoning is just 
that of Example 4.4.1 except that & is taken as the above subset of & 
bounded by Z and @. 

The fact that the optimal strategies of P—that of assuring termination 
and that of minimizing the numerical payoff—are distinct has little effect. 
He never need be concerned with the former unless x gets very close to 2. 
(As there is no lower limit to this closeness, we might say the latter 
strategy is optimal throughout the interior of &’.) But should E be foolish 
enough to make some attempt at escaping, P can wait to frustrate him until 
x gets very close (if it does) to Z, meanwhile playing the game of degree 
optimal strategy; E will not escape but will be penalized in payoff also. 

Again we have part of the formal solution V(x) (to the right of B) not 

being the Value. But the main point here relates to the final clause of the 
theorem. We sketch another such instance. 
Example 4.4.5. The same as Example 4.4.1 except for the inequalities on w. 
Far to the right in £, w(x, y) is subject to the earlier strictures, but as x 
decreases, w grows until at the far left w > 1 and w > u. From such 
starting positions P has command of the motion of x; he can avoid 
termination if he wishes. 

Let it be that he does so wish and E seeks it when possible. Such is 
the case sufficiently rightward in &. 

Again it is possible to segregate the termination and escape points by an 
SPS of the type which in Chapter 8 will be called a natural barrier. Any 
premature details here would fill an undue amount of space. 

Once this barrier is known, the discussion becomes very much like that 
of the last example and will not be repeated. 


B 


E 
Figure 4.4.4 
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The only point we wished to make here is possibly superfluous. In all 
the previous examples the question ranked strong of the inevitability of 
termination. Only in 4.4.4 did we admit the alternative possibility. Did 
our means seem artificial there? First, assurance by bounding the down- 
ward velocity of x from below; then negation, by cutting off @. We 
wished to show here that our ideas stand up under “natural” circumstances 
too. 


4.5. THE PATH EQUATIONS 


We now revert to the technique for obtaining solutions. The contents of 
this and succeeding sections will be used extensively throughout the rest 
of this book. 

Our starting point will be the ME,, which we rewrite for reference: 


E VIX $, P + G& $, p) =0 (4.2.3) 


where ¢ and # are functions (4.2.2) of the vectors in x and V, 

We differentiate (4.2.3) with respect to each z,. Doing so according to 
the rules of elementary calculus, we examine the components as they 
arise. First we have 


ov, 
— fi. 4.5.1 
2 FPE GSN 

; ov, eV ov, - 

— = —— = — d = Tj 
But since aa olda Dn and f, = ż 
04.5.1) = 5 24a, = Vy 
T Ot 


The latter term would be the time derivative of V along the resulting path 
were ¢ and @ played. Next we have 


E Vafa + Gr 


Ox, 


; 0G 
where f;,(x, u, v) = we (x, u, v) and G, = =—. Then we encounter 
k 


$A (sus+ ot. (4.5.2) 
iah \ i GEA 

Each ¢, is supposed subject to constant bounds as discussed in Section 

2.7. The minimizing 4, occurs either interior to the constraining interval 

or at an endpoint. If the former, the 0/04, (-) term of (4.5.2) is 0 because of 

the minimizing property of $; if the latter, the 0¢,/02, is 0 as $; is constant. 
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In either case then (4.5.2) vanishes. The same is true of the remaining 
terms devolving on the #,. We conclude 


Pa = -SF Kifa Hox, Ve), He, V) + Gale, Hx, Va), HO, Vol}. 45.3) 
Using the barred control variables in the KE themselves gives further 


Ly = falx, Hx, Va) wx, V,)I. (4.5.4) 


This set (4.5.3), (4.5.4) of 2n ordinary differential equations in the 2” 
unknowns 2,, V, shall be called the path equations. Actually they are the 
characteristic equations! of (4.2.3) (slightly special in that the terms (4.5.2) 
are nullified). Solutions of the ME, can be built from integrals of the path 
equations in the standard manner,’? a procedure we shall shortly adapt to 
our purposes. 

Note that the brace in (4.5.3) is nothing more than the formal derivative 
of (4.2.3) with respect to 2,, that is, we ignore all but the explicit appearances 
of x, in (4.2.3). 


46. THE RETROGRESSION PRINCIPLE 


When solving a game we reverse time; we start at @ and work back- 
ward into &. From the point of view of differential equations the motive 
is clear. In the previous section we effectively reduced the construction 
problem to the integration of the path equations. We cannot obtain the 
appropriate particular integrals without initial conditions. And at the 
outset of a problem we will have the needed data only on @ where we know 
V= H. 

But perhaps a stronger, if more heuristic, insight can be gleaned from the 
discrete examples of Chapter 3, where building backward from the terminal 
states was plainly the only means at ọur disposal. And it would appear that 
any differential game can be quantized into a discrete one. 

We shall henceforth use the symbol 7 for the time needed for x to reach 
€ (or some other surface playing a like role) so that on any optimal path 


7 = constant — t. 
The symbol will mean 0x/@7 so that 
° . 
t= =t. 
The path equations, when rewritten in this retrogressive notation (the 


12 See, for example, Reference [6]. 
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signs of the right sides are changed), will be referred to as the RPE ( Retro 
gressive Path Equations) and will be an essential constituent of oy 
standard procedure: : 


= hos $, P) 4.6. 
È, = E Vafa, $, P) + Gx, $, P). (4.6.2) 


Again we note that the right side of (4.6.2) is the derivative of (4.2.3) 
with respect to explicitly appearing xy. 4 
Let us recall that (4.6.2) was derived on the assumption of constang 
constraints for the control variables. Although such can always bg 
brought about, occasionally problems arise where it is inconvenient tq 
do so.!* In these cases we must adjoin terms to (4.6.2) corresponding t@ 
suitable derivatives of the proper control variables. i 
It is interesting to note that the RPE are Hamilton-Jacobi equations; 
although we have found no means as yet of capitalizing on this fact4 


Exercise 4.6.1. Find the RPE for the ME, given in Exercise 4.2.1. Takél 
u=1-+ 2, w= $e”. Verify that the left side of the ME, has zero 
derivative with respect to 7. 


Exercise 4.6.2. Find the RPE for the homicidal chauffeur game in the 


realistic space. The ME, is here 


w,(V, sin 6 + Va cos 8) + wap + Vad + 1=0 


where p=VV2+VeZ, $= —sg V, 


and ipea. cos ğ = +. 
P 
The KE are on page 28. 


(The answers to these two exercises appear at the end of this chapter.) $ 


Problem 4.6.1. From the solution to the last exercise, show that for the 
solution in the small of the homicidal chauffeur game, P always turns $ 
sharpest possible right or left and E travels in a straight line. Thus we can q 


expect the solution to have many singular surfaces. 


4.7. THE INITIAL CONDITIONS 


We use this term in the retrogressive sense. We are concerned with d 
known values of x and V, on @ (or some other surface playing a like role) @ 
which can serve as initial conditions with respect to 7 when we integrate ‘i 


48 For example, Example 5.6. 
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RPE. But in the progressive sense, that of ¢, the way a partie actually 
svelops, they are final conditions. 

Jn many games not all of @ is capable of serving as a receptor for 
rmination. In the homicidal chauffeur game, for example, if we regard 
„as the planform of the car driven by P, it is clear that only the front 
umper will be effective; it is rather difficult, when driving forward, to run 
ver a fleeing pedestrian with the rear of one’s car. 

To study this phenomenon in general consider a position very near @, 
One or the other player may be able to force or deter an imminent termi- 
pation despite any opposition from his opponent. Let » = (7, .. . , »,) be 
the vector normal to @ from point x on @ and extending into &. If 


min max > », f(x, $, y) > 0 (4.7.1) 
Y i 


- then E can prevent immediate termination from a position sufficiently 


near x. If (4.7.1) holds with the inequality reversed, P can compel im- 
mediate termination. 

There is the question of whether a player will benefit from the exercise of 
such power. Sometimes the answer is obvious. We cite the case of 
termination time payoff; clearly E will defer termination whenever he can. 
But in other instances E may see that avoidance of the frying pan now will 
only lead to the fire later. We leave the intricacies of such questions to 
individual cases. However, in all yet encountered there have been no such 
difficulties. For definiteness, let P desire termination and let it be to E’s 
advantage to avoid it if he can. Then we have found that whenever ( 4.7.1) 
obtains, E will defer an imminent ending. 

Similarly, at those points of @ where (4.7.1) holds with the inequality 
reversed, P will occasion termination immediately. These points will be 
called the useable part of @. Termination will occur only at the useable 
part under optimal play. 

The subset of @, where (4.7.1) holds as it stands is the nonuseable part. 

The curve [(n — 2)-manifold] on @ which separates these parts, that is, 
for which 


min max $ v»; f(x, $, y) = 0 (4.7.2) 
$ v i 


is called the boundary of the useable part, abbreviated BUP. 

Of course, in many problems all of & is the useable part, for instance, 
those which include a kinematic equation, 7 = —1, and ¢ lies in the 
plane where T = 0. But in many others, ascertaining the useable part is 
an early and important step in the solution. 

The initial conditions we need for integrating the RPE are then the 
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values of x, and V, (i= 1,...,”) on the useable part. The parametric 


representation of @, 
x; = AfSy,. ~~ Sp) (4.7.3) 


when restricted thereto, gives us the first n. To obtain the V, on the 
useable part we recall that on &, V = H = H(s,,..., 5,3). Differenti- 
ating with respect to sẹ, gives us 


OH syth k=1,...,n—-1 (4.7.4) 
Os, T Os 
a set of n — 1 equations for the n unknowns V,. The remaining equation 
is the ME, with x, replaced by (4.7.3). 

Sometimes a double solution to the set will appear. The reason is that 
there is nothing in our analysis to distinguish the two sides of . For 
example, in a pursuit game, @ will often bound a region corresponding to 
capture. We are interested only in x’s encountering @ so as to pass from its 
exterior to this interior capture region. But a game with the reversed 
crossing of @ as termination also has a valid interpretation." 

In working problems there is always some simple way of telling which of 
the two possible solutions to retain. As a workaday matter, we warn the 
novice to be careful; here is an easy point on which to err. 


Exercise 4.7.1. Obtain the useable part for the homicidal chauffeur 
game, using the reduced space. For @ take the circle of radius /: 


x = sins, y = l coss. 


Find the initial conditions on the useable part both for paths which 
emanate from @ to its exterior and interior. 


Our technique for solving problems in the small is now virtually com- 
plete. After integrating the RPE with the initial conditions just explained 
we obtain the 2n functions x, and V, of the n arguments 


T, Sty 00 oy Spt: (4.7.5) 


We should now invert the first n functions and solve for (4.7.5) as functions 
of the z,. (This is at times a formidable formal elimination, but in most 
particular cases we can get around such difficulties with a bit of ingenuity.) 
We can then find V either by inserting the newly found functions in the 
second n integrals, thus obtaining V,(2,,...,2,) and then integrating 
these to get V to within an additive constant, which is fixed by V’s known 


14 If the interior of @ was the set of points under surveillance by a detection 
device, such as radar, carried by P and E’s objective is to slip out of range and P’s 
to prevent him. 
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value on @, or we can work directly with fG dt + H. The optimal strategies 
are known by inserting the v, and V,, after the elimination, into the 
Hx, Ve) and #(x, V,), which accompanied the ME}. 

Of course, the solution in the large entails the study of singular surfaces, 
and many subsequent chapters will be devoted to particular types. 


Answers 
Exercise 4.6.1. 
i= +per, 7, = ay, 
P 
9 ~8y Vy P 3 ~By 
y=1+ te" V, = ~ze "p 
P 


where p = sgn V,, p=v V+ Ve 
Exercise 4.6.2, 


#,=—w,sind V,=0 
J, = —w,cos6 Vy =0 
Pia yt y, = 0 
P 
ears V,=0 
p 
o — W. o + 
0 = — $ V; = w,(V, cos 6 — V, sin 0) 


—sgn V;, p= V Vz + v 


~~ 
oH 
I 


% 

















CHAPTER 5 


Mainly Examples; Transition Surfaces; 


Integral Constraints 


We shall put the theory developed thus far to work by solving several 
examples.’ Essentially the only type of irregular behavior occurs on what 
are termed transition surfaces. The section immediately following gives 
an account of these singular surfaces adequate for the ensuing problems, 

Our opening example parallels the debut of the calculus of variations. 
In 1696 John Bernoulli challenged the mathematical world with his problem 
of the brachistochrone or curve of quickest descent. He, Newton, Leibniz, 
L’Hospital, and Euler all reached solutions. The variational calculus was 
born. 

A weight, subject to frictionless, constant Newtonian gravity, descends 
from a given starting point to a second lower one. If it is constrained to 
some curved path, which one minimizes the time of descent? The 
destination may be a curve instead of a point. 

Instead of viewing the falling object as constrained to a scheduled route, 
we may think of it as being steered at each instant by a pilot who can 
choose the travel direction freely. For Newtonian mechanics governs only 
the speed—the familiar v = y 2gh—and exercises no other jurisdiction on 
the motion. Thus the pilot may always choose from a circular vectogram. 

There is no reason why we cannot imagine a second player, with a 
second superposed vectogram, whose object is to maximize the time of 
descent. We do so and are in the realm of game theory with the problem 
of the dolichobrachistochrone, the curve of slowest-quickest descent. 

Section 5.4, following a study of this problem, contains a generic 


1 This chapter is a re-edited version of Rand Report RM-1486 (25 March 1955), 
entitled Mainly Examples, which does not include the section on integral constraints. 
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answer to the question provoked. What is the relation between our subject 
and classical calculus of variations? l , l 

The second problem is a simplified, primary version of a major question 
of military policy. In a protracted war, how should a combatant split his 
efforts between the long-range objective of attrition—destroying the 
enemy’s sources of weapon supply—and the more immediate one of 
directly attacking military targets. Simple as is our model, the answer is 
not an obvious one, quantitatively at least. In Chapter 11 we will Te- 
sume this problem with a discussion of its relation to reality, and Section 
11.9 will contain a modified version which has a strikingly different 
solution. ; 

The isotropic rocket problem of Section 5.5 is a pursuit game in which 
the pursuer, who steers by control of the direction of his driving thrust, 
chases an evader with simple motion. Like the homicidal chauffeur 
model, there is the possibility of a “swerve” under certain conditions, but 
unlike it, the solution is generally smooth and analytic rather than segre- 
gated by a multitude of singular surfaces. The solution appears to be a 
mélange of the straightforward and the unobvious. 

The final problem is an application of our techniques to economic 
programming. One-player games can be treated by thinking of the 
opponent as inactive; his range of decision is null. What remains is a 
problem of direct optimization. The present instance is an idealized 
picture of steel production in which the current supply is allocated between 
use as an ingredient in making more steel, building more steel mills, and 
the stockpile. What such allocation at each instant maximizes the steel 
supply at the termination of the program? 

The chapter ends with a technique for problems where one player at 
least is subject to a constraint which can be expressed as an obligation to 
keep constant an integral over the path of a function of x, ¢, and y. 
Such is the counterpart here of a classical side condition in the calculus of 
variations. Lack of space limits us here to but some new versions of a 
famous problem, but the idea is used more fruitfully in the bomber and 
battery game in the Appendix. 


5.1. TRANSITION SURFACES 


This, possibly the simplest and most direct, type of singular surface can 
arise only in conjuction with linear vectograms. Let one of the control 
variables, say ¢,, appear lineally in each KE and G (as defined in Section 
2.4) with coefficients that are independent of the others. Then, when we 
construct the ME, ¢, will appear lineally there too; its coefficient A will 
involve at most the x, and V,. 


% 
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Let a and b be the constant constraints on ġ;: 
agg <b. 


Suppose the solution were known at a point x of @. Then A is known there, 
and so supposing ¢, to be minimizing, we would have at x 


$, =bifA <0 
¢, =aifA>0. 


For definiteness, let us suppose the former holds throughout some 
neighborhood of x. Along the optimal paths through these points, 
$ı = b as long as A < 0. But suppose there occurs, as we follow each 
path (increasing 7), a point where first A = 0. Normalcy implies that 
these points constitute a surface 7. 

To ascertain the solution beyond 7 we can use this surface as a seat of 
initial conditions, that is, it plays the role of @, and our standard pro- 
cedure will yield optimal paths emanating from it. For initial conditions, 
we can use the values of the x; and V; resulting from the integrals of the 
RPE that led to 7. (of course, V can be also computed on J and then 
employed as H there. If we were to apply the standard procedure of 
ascertaining the V, on 7 in terms of the partials of H, it is not hard to see 
that the same V, as above will result.*) 

On the new paths, ¢, will be determined by the local sign of A. As it is 
zero on J, at the outset we must rely on A. The latter can be found by a 
simple calculation which will actually be carried out in general in Section 
7.4, It turns out that Ais independent of 4. 

Now A will be 0 on 7 only for a special class of surfaces which are 
detectable in advance and will occupy us extensively in Chapter 7. 
Therefore there exists grounds for the assumption that 4 #0. This 
question will recur in better prepared context in Section 7.11. 

, We can conclude that when an optimal path crosses 7, where A = 0, 
A #0, A changes sign and so ¢, shifts abruptly from one of its extreme 
values to the other. Thus the name transition surfaces for such as 7. 


5.2. THE DOLICHOBRACHISTOCHRONE? 


In the classical brachistochrone problem a body in a uniform gravi- 
tational field is constrained to slide down a given curve. Its starting point is 
prescribed and so is its terminal position, which may be a second point or, 


2 This calculation also entails the ME,. But as A = 0 on J, it is immaterial whether 
we use it with $, = a or b. 

3 In this chapter, where examples dominate, example and section numbers will 
coincide. 
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more generally, somewhere on a given curve. The latter alternative 
accords more closely with our ideas; the body is to end its fall at some 
point of a specified curve g. We do not know which point and ascertaining 
it is part of the problem. 

The issue is to find the constrained path which renders minimal the time 
for the body, if it starts from rest at the given starting point, to reach @. 

If from a stationary start, the body has descended a vertical distance y, 
no matter what its path, elementary mechanics tells us that its speed: will be 
/2gy. Only thus does classical mechanics enter the problem. Since it is 
clear that the sought curve—the brachistochrone—is independent of g, the 


gravitational constant, we may take the speed as being Vy. 

In our framework, we think a point capable of being navigated about 
the upper half of the x, y-plane in such a way that its direction of travel at 
each instant is under our control, but its speed is always Jy. Itis clear that 
if we navigate so as to minimize the flight time, the situation is tantamount 
to the above; the optimal route will be the brachistochrone. 

Thus, in the language of differential games, we take as KE 


é = Vy cos $ 
gj = Vy sing 


so that, as we wish, the speed of x = (x, y) is always J y, but travel 
direction of inclination ¢ is always at our—or rather P’s—disposal. At 
each point he has a circular vectogram. Further, an integral payoff with 
G = 1 leads to the minimization of the transit time. It is trivial to observe 
that the usual convention of coordinates associates positive y with an 
upward direction; accordingly we have tacitly reversed gravity and will 
not be disconcerted by the ‘“‘body’s” falling upward. 

As our terminal curve @, we will select the positive y-axis. The playing 
space & will be the first quadrant of the plane (x > 0, y > 0). Any point 
of & can be used as a starting point, providing the “body” or point x is 
thought of as having an initial speed = Vy. The positive x-axis is then the 
only set of starting points conforming to original framing of the problem. 
If the stationary start is important, we can always attain it by the proper 
choice of axes, but rather let us accept the modified starting rules.* 

In the dolichobrachistochrone problem, the second player E strives to 
maximize the time through the addition of another velocity to those of the 
vectogram already discussed. He makes his selection from the vectogram 
shown at Figure 5.2.1a. The two extreme vectors are each of length w. 


4 Sometimes the classical problem is so stated that there is given initial speed along 
the curve. This is achieved by using a starting point with a suitable y. 
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Figure 5.2.1 


One gives him the chance to delay the progress of x toward @ by pulling 
straight downward, thus impelling x into the region of small y and hence 
of low speed available to P; the other extreme vector is horizontal and 
directly repels x from @ [(b) of the figure roughly depicts a typical instant 
of a partie]. The total choice open to Æ is a linear convex mixture of these 
extreme vectors and so is typical of a linear vectogram. Thus the question 
confronting him concerns the circumstances warranting the most profitable 
combination of striving directly to draw x away from @ or, less directly, 
seeking to retard x by driving it deeper into the region of low speed. 

Of course, there is a game of kind here too. For y < w*, E’s speed 
dominates that of P and the latter cannot force termination. In fact, by a 
suitable alternation of his extreme vectors, it is not hard to see that E can 
force x as far away from @ as he pleases. As far as possible, we shall ignore 
this aspect here. 

In the following analysis, the reader may prefer to think of w = 0 
throughout. The result will be the differential games version of the classical 
brachistochrone problem. 

The KE are 


t= Jycosġ +5 (y +1) 
ġ = Jysin g +> (y — 1), -1< p<1. 


5 It will be discussed in Example 8.6.4. 
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With the final terms elided, these equations have already appeared. 
The pair of final terms themselves are 


(w,0) wheny=1 
and. (0, —w) when y = —1 (5.2.1) 


so that for the extreme allowed values of y we have the extreme velocities 
of E’s vectogram. It is clear that intermediate y complete it in the manner 


described above. 
As time to termination is the payoff, the latter is integral with G = 1. 


The ME, is accordingly 
min max Ra cos $ + V,sin $) + Saly +1)+ Vy — 1) + 1] =0 
$ Y 
The min is obtained through Lemma 2.8.1. Putting 
p= VV,2 ees 


the min is furnished by $, where 


x 


æ 


F TEE z 
p 


P 
and the first round parenthesis in the ME, becomes —p. 
Since the coefficient of 4wy in the ME, is 


A=V,+V, 


sin $ = — 


the max is attained by 
p = sgn A. 
Thus the ME; is 


-Vy p + SAG + Va- V) +1=0 


or 
—vVyp +t wV, +1=0 whend>0 
and —vVyp — wV, +1=0 whendA <0. 
The RPE are, when A > 0 (Ẹ = 1), 
t= JE —w, V,=0 
P 
o — A d 
y= VI YW=—- >>>: 
var are 
When A < 0 (Ẹ = —1) the —w term is removed from the equation for £ 


and +w is added to that for y. Such is clear from a glance at the KE. 
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We parameterize @ by 
z=0, y=s2>0 
V(=H) = 0. 
To find the useable part, we first note that the normal velocity to @ is 
horizontal and so is given by ż from the KE with x = 0, y = s. For 


penetration of @ to be possible by P, against all opposition from £E, the 
minimax of this velocity must < 0 and so we must have 


and on @, 


min max @ = min max [vs cos ġ + Ë (y + D] 
$ v $ v 2 


=—/s+w<0 
or the useable part is that part of for which 
s>w. 


To complete the initial conditions for integration of the RPE we must 
know V, and V, on the useable part of @. There we have 


V= H,) =0= Via, + Vis = V, 


On @, V, > 0. For V = 0 on @, and we must have V > 0 a little to the 
right of &, as is clear if we recall that the meaning of V is the time required 
to reach @. Thus A = V, + V, = V, > 0 and so near and at @ we may 
take # = +1. The ME, at @ is then (we put V, = 0; as V, > 0, p = V,) 


-Vs V, + wV¥,+1=0 
or on @ 
1 

Js—w 
which is positive on the useable part. This equation, V, = 0, and (5.2.2) 
are the initial conditions. We employ them in the integrals of the RPE. 

The upper right RPE integrates at once to (5.2.3), which is now con- 
strued as holding in at least some of @ rather than merely on @. The 
easiest way to proceed with the integration is to utilize the ME, (with 
y= I): 


V; = 





(5.2.3) 








(5.2.2) 1 
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p’s higher speed renders the time (= V’) to reach @ smaller and so V, < 0. 
A more formal criterion we leave to the reader as 


Exercise 5.2.1. Show that to satisfy the lower right RPE, the + in V, 
must be —. 
Noting that _ 
S 1 
p= x ‘= 
J- Jy 


the lower left RPE becomes 


= a(-./£-1)- sandy -# E: 


which has as the integral with y(0) = s 





y= S(1 + cos =) (5.2.4) 


as may be readily verified, provided that 
T L mys. (5.2.5) 


Nothing but standard methods are required to integrate the upper left 
RPE; the result is 


x= s + sin — — wr. (5.2.6) 
Js 
The optimal paths are koe by (5.2.4) and (5.2.6). If w = 0, we remark 
that they express the classical cycloids normal to @. The generating circle 
here rolls on the z-axis, has radius = s/2 and rotates through an angle 
rvs in time r. The inequality (5.2.5), necessary for y in the RPE to 
maintain the proper sign, precludes cycloidal arcs of more than a half 
revolution (x cannot reach and then leave the x-axis). 
Returning to the two-player game, we have A > Oand so E plays Ẹ = 1, 
his horizontal extreme velocity vector. Let us see how long this state per- 
sists. We see that 


A=V,+V,= -0-y -D 
Qs — w) 
and A remains positive as long as y > 4s. When y = $s, we should expect 
a transition surface (TS) where E shifts his strategy from ọ = 1 to ğ = —1. 


From (5.2.4), on the TS, cos TN. s = 0 and from (5.2.5) this implies 





=Y l (5.2.7) 
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Figure 5.2.2 


Then on the TS, from (5.2.6) with 7 = To, 


ga E + 1s — wos. (5.2.8) 


Note that in this domain of the first stage paths (prior to the TS) 
Ver. (5.2.9) 


The equations of the TS are (5.2.8) and y = 5/2; on it V = m = 
(7/2)\'s. It is thus a parabola with inclined axis and appears as in Figure 
5.2.2. It delineates E’s optimal strategy; he always plays one of his 
extreme velocity vectors depending on which side of the TS x lies, as 
marked on the figure. 

The optimal strategy of P is expressed in terms of V, and V, as in the 
derivation of the ME,. To the left of the TS, for example, 


ve _ vy 


P Js 


But a formal expression in terms of x and y is awkward. We would have 
to solve the path equations (5.2.4) and (5.2.6) for s and 7 and use the value 
obtained of the former in the above formula. 

The optimal paths to the left of the TS, as we can see from their equations, 
are generated somewhat as are cycloids. Each is described by a point on 
the rim of circle rolling to the right on a platform sliding to the left with 
speed w. 


cos $ = — 
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To proceed with the solution to the right of the TS requires some 
further integration, which, although elementary in principle, appears 
rather oppressive. As initial conditions we would have 


c= jbo 





and our task would be to use them in an integration of the RPE with 
g = —1. The ME, here would be 


—pV'y — wV, +1 =0. 


There seems to be insufficient reward for the tedious formal integration 
required. 

To establish that such a calculation actually completes the solution we 
would need to know that 


1. the new paths cover the subset of & to right of the TS and for which 
y > w? completely and univalently. 
2. A < 0 on this domain (except at the TS where A = 0). 


Now 1 seems highly plausible (a proof without integration is probably 
not too hard). And 2 follows from 1, for along an optimal path 


A=-— <0. 
2/y 
5.3. THE RELATIONSHIP TO THE EULER EQUATION 


The connection between our approach and the classical one, illustrated 
by the last example, when w = 0 will be discussed here in rough terms. 
We deal only with planar one-player games with integral payoffs and a 
single control variable; more elaborate cases can be inferred from this 
instance. 

We start from the KE 


& = filz, y, p) 


9 = fz, y, $) 
with 


foe, Y, $) dt (5.3.1) 
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to be minimized. We can write (5.3.1) as 


Gz, Y, P) ay 
Fi, Y, $) 


Ha 


and also 


We solve (it is generally possible) (5.3.3) for ġ and substitute the result $ 
in (5.3.2), which now assumes the familiar calculus of variations form: $ 


fr (x, y, y') dz. 


The case where F is independent of y’ plays a special part in our theory, 


and will reappear during our later discussion of universal surfaces. 


On the other hand, if we start from (5.3.4), there are many ways of. 
reaching: the KE and (5.3.1) inasmuch as (5.3.4) takes no account of: 


the /-parameterization. The simplest appears to be 
č = cos ġ 
ý = sin ġ 
G = F(z, y, tan $) cos ¢. 


5.4. THE WAR OF ATTRITION AND ATTACK 


The two nations, P and E, engaged in a protracted war, have respective 
supplies of a vital weapon, x, and z, at time t. Each has at all times the 
choice of how to allocate his stock between “attrition,” that is, depleting 
his enemy’s rate of weapon supply, and “attack,” that is, entering them in 
the major conflict. It is the accumulation of the latter entries that count; 
each player seeks more than his opponent, and the excess will be the payoff. 

Thus the basic decisions here are between the long-range policy of 
attrition and the short-range one of direct attack on the essential targets. 
We shall formulate what appears to be the simplest version possible of this 
broad problem. It will be discussed further in Chapter 11, where we shall 
frame an alternative model in Section 11.9. 

The realistic execution of the strategies would comprise a series of 
discrete decisions. But we shall smooth matters into a continuous process. 
Such is certainly no farther from truth than our assumptions and therefore 
can be expected to be as reliable as a stepped model. It also is more facile 
to handle and yields general results more readily. 


€ Proposed by Arnold Mengel. 


E a 


(5.3.4) i 
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Let P, at time ¢, split his force x, into the attacking component (1 — ġ)xı 
and that of attrition ¢x,. Here ¢, the fraction devoted to the latter purpose, 
satisfies 0 < $ < 1. If E is unimpeded he has the capacity to manufacture 
weapons at the rate m,. He also loses them at a rate depending on the 
number $x, his enemy is devoting to that purpose. For lack of better 
information, let us take this rate as proportional to the number. Then we 


write : 
may Žž = Mm, — ca($x) 


where the coefficient c, may be regarded as a measure of effectiveness of 
P’s weapons against E’s defenses. 

By reversing the roles of the players we obtain a second similar equation; 
they will be two of the KE. 

Let us suppose we plan on the. war lasting some definite time T.” Each 
day (let us say) the combatants put into the field the numbers of weapons 
(1 — ¢)x, and (1 — y)x,. The sums of these quantities for each day will 
reflect the respective sides’ total battle strength, and difference of such 
sums, the margin of superiority. Such will be the payoff except that, in 
accordance with our policy of smoothing, the sum becomes an integral: 


T 
[ta ~ vee a- paat 


To fit the framework of differential games we adopt T (or 2, if preferred) 
as a state variable. Then the totality of the KE is 
i, = M, — Cpe 
ty = M, — Co, 
T=%4=~-1 
with 0 < 4, y < 1 and G = (1 — y)x, — (1 — pje. 
The space & will be the octant 


Tı > 0, 
and @ will be its partial boundary where T = 0, which may be parameter- 


ized as 
zi1=Ss 20, %2=s5s,>0, T=0 


z> 0, T20 


these equations being part of the initial conditions. 


We shall also suppose 


Cy > Cy (5.4.1) 


reversing the notation if the contrary were true. 


? Alternatives to such assumptions are suggested in Chapter 11. 
*In this type of problem we prefer to eschew such symmetries as c, = c as being 
too coincidental. 
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For the time being, we shall ignore the possibility of x, or £ becoming 4 


negative. In this way we can get to the heart of the problem with less 


encumbrance. 
Denoting the partials of V by V,, V2, Vip, we can write the ME: 


min max [(m, — cyx_)V, + (mg — C2921) Vz — Vr 


$ y 
+ (1 — pa, — (1 — $)x,] = 0 
and the ME, is then 


Sinh + Soto + mV, + mV — Vr +%,—2,=0 

where S1 = 1—cC.¥, S = —1 — cV 
OifS,>0 
~ Lif Ss; <0’ 


Oif S, <0 


P = \Lif Sp > 0. 


and $ 


Observe the tacit assumption that 2, x, > 0. 
The RPE follow: 


EA = —M; + Cpt, Vy = Sid -1 
Z, = —m, + cft, V, = Syptl 
T=1 


the expression for Vp turning out to be superfluous in this particular game. ` 


Let us complete the initial conditions. As on @ we have V = 0, 


dv, ôT 


Ou 
hae Vig tha. Pie" 
and, similarly, V, = 0 on @. There, then, 
S,= 1 andso ¢=0 
S= — 1 andso ~=0. 


Thus the war concludes with both sides fully attacking. 
We now integrate the RPE, using the above initial conditions with 


$=p=0. 


xy = $1 — MT V= rT 
Tz = Sg — MoT V=r (5.4.2) 
T=7. 
On the optimal paths 
S, = 1 — cr, S, = —1 + c7 


and these first cease to be positive when 7 = 1/c, and 1/c,. Because of 
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(5.4.1) the latter occurs first (retrogressively). Thus we should expect a 
TS (transition surface) when 
(r=) T= - (5.4.3) 


where # should change from 0 to 1. Such is the case; continuation of the 
analysis furnishes a proof. Then at a time 1/c, short of the end of the war, 
E switches from full attrition to full attack. 

Let the surface (5.4.3) be called 7,. The Value between @ and 7, is 


given by 


T r T 
y= | 0 at= f" tea — 2) dt = [Use — mar) — Ci — ma] d 
= (S2 — 8,)T — }(m, — m,)T”. (5.4.4) 


To express V in terms of the state variables, we put r = T in (5.4.2) on 
the left and eliminate s,, Sẹ. We find that 


V = (£, — %)T + Hm, — m )T?. (5.4.5) 


To continue our analysis, we treat 7, as a seat of initial conditions in 
the same manner as we did @. We may either place r = 1/c, in (5.4.2) and 
again use sı and sz, but now as parameters of 7, or we may start afresh 
with new parameters. Choosing the latter alternative, 7, is 

1 
Ti = Sy Ta = Sp T= — 
Cy 
where these:s; are not the same as the old. From (5.4.2) on the right we 
have at once that on F, 


Of course these two conditions could also be derived in our standard way 
by using (5.4.5) on 7, as H. 

When integrating the RPE with the above starting data, we shall use 
¢ = 0, y = 1. This usage will be confirmed if it leads to S; of the proper 
signs. The reader may find it beneficial to perform the calculation which 
results in 

z, = sı + (C18, — m )r — amg? 


T = Sg — MoT 


(5.4.6) 


(5.4.7) 
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Here 7, as the s;, was chosen afresh as 7, and is not the same as in the 
earlier integration. We find now that 
c 1 
S = 1 — Ž — egr — = eyegr* 
Cy 2 


S = cyt. 


Clearly S, > 0 when r > 0, and we are confirmed in our expectation 
that p = 1. 

When + = 0, because of (5.4.1), S1 > 0 and so ġ = 0 for small 7. But 
this condition will cease should S, stop being positive. The equation, 
S, = 0, has just one positive root, which is 


as EER EAE) 


If we accept this value as marking a second TS and also (very plausibly) 
assume there will be no further shifts in the control variables, we have 
fully found the optimal strategies in those cases (of most practical interest) 
where x, and x, do not become zero. 

At time 1/c short of the duration E shifts from attrition to attack and 
P does so earlier, at a time preceding termination by 


À Jala) — 1. (5.4.8) 


The last value arises, of course, because of the new TS, which we will 


call Z, when T = (5.4.8). 
Let us now find V between the 7, and.7,. We can do so in two ways. 


First we can write 
T—-1/c1 
Vey — Í x dt 
0 


where V™® is V on J, obtained from (5.4.5) with arguments from (5.4.6). 
The integrand x, (= G) is also taken from (5.4.6), and finally we use these 
equations to eliminate s, and sz. 

The second way is to take V, and V, from (5.4.7), replacing 7 by 
T — 1/c,, and then getting Vy from the ME,. Integrating these partials 
gives V to within an additive constant, which can be ascertained by its 
known value on J. 

The result is 


3 
paote (= ai z)T+ Heisa — m)T? — an, (5.4.9) 
2 
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Observe that beyond 7,, G = 0 and there are no further changes in V. 
It is then given by (5.4.9) with T fixed at (5.4.8). 

We can now accept on heuristic grounds that 7, is a TS and there will 
be no further shifts in strategy, or use the criterion to come in Section 7.11, 
or continue with the solution beyond 7,. We can then see directly 
whether there are further sign changes in S, and S}. We also obtain the 
optimal paths in this domain. 

We leave the decision and, in the latter case the effort, to the reader. 

There remains the embodiment of the constraints x, > 0 into the solution. 
The best method seems to consist of first treating the boundary parts of &, 
where x; = 0 as 2-dimensional subgames; when the Values of these are 
known, they can be used as H and new optimal paths into & can be 
constructed which may be merged with the old. 

More specifically, let @, be the subset of the boundary of & described 
and parameterized by 


1 =0, %=5 20, T=5s,>0 


and @, is defined similarly with the subscripts 1 and 2 interchanged. 

Now when the state of a partie (that is, x) is on @,, P has no weapons 
(2, = 0), and so it would be futile for E to induce more attrition (take y 
larger) than is needed to keep x, at 0. It is of course futile in practice 
because E would be wasting weapons on attrition that could be used for 
attack and so score in the payoff; but in theory, when we view the game as 
a purely mathematical problem, we might regard such a limitation on y as 
a given constraint necessary to keep x within the bounds of &. 

Let us consider the part of @, or G, lying between @ and Z ,, a region 
where we know that $ = # = 0. From the KE, #, > 0,4, > 0 (intuitively: 
the weapon numbers increase with no attrition); therefore at these points 
the optimal paths leave @, and @, (progressively) and present no problem. 

Figure 5.4.1 is a typical cross section of & for some constant fairly 
large xy, with the optimal paths indicated. The paths found earlier 
between @ and 7, appear as those above AB; those below have just now 
been discussed. If they are traced back (increasing 7), they each reach ¢, 
at some point of OA and induce no modification to our analysis. 

But on @, beyond 7,, where p = 1, it can be that x, remains 0 during 
an interim. There can be paths like CDEAB on the figure. Recalling the 
KE 

ty = Mı — Cyp%s 


we see that x, can remain 0 (such as on EA, as just illustrated) only if 


a>, (5.4.10) 
Cy 
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x1 
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À 
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SZ 
LADS 

oS E : 5 — T 
oO o=0 1fey o=0 @ g=1 
y=0 y=l vel 

Figure 5.4.1 


When this is so, and x, = 0, E instead of playing y = Ẹ = 1, should | 
employ | 
m (5.4.11) ` 


Cita 


y = 


until x reaches 7, N @, (A in the figure). For this phase we set up the 


subgame as follows: f l 
The state variables are. x, and T (as x, = 0); the temporary @ is subject 

to (5.4.10) and T > 1/c,. (It appears as the hatched region on the “‘floor’ 

of Figure 5.4.2, which depicts the & of the original total game.) The : 


temporary @ is given by 


ahaa hy eimai REN ie 





bli ates 


pa 


m 
%_ = o (a parameter) > — 
Cy 

om 

Cy 


and H is the Value (5.4.5) at these points: 


Hal 4 =m) 


2 
Cy 2c, 
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The KE are 


ža = M, 
t= =i 


and for G, in virtue of (5.4.11) and z, = 0, we have 


G= (1-72), = 2 — ™. 
Cy 
We leave the details of this simple subgame, with no control variables, to 
the reader. All we will utilize of its solution is, as yielded by certain 
integrals of the RPE, 
Ve =T 


Vr = — T+ ay + maT. 


1 


(5.4.12) 


Now we return to the original game. We are to construct regressive 
paths with partial initial conditions on part of @,: 


x, = 0, a = s > E, T= s; > =. 
Cy Cy 
Putting these values into (5.4.12), we obtain further initial conditions. 
The remaining V, is obtained from the ME, with ¢ = 0, y = 1, the result 
being 


The integration of the RPE with the above initial conditions is done as 
earlier in the section. Similarly, as earlier, we investigate sign changes of 
the S, along these paths to locate TS. 

We shall omit the calculations. It turns out that S, remains positive, 
but Sı changes sign, but not on 7,. Rather, part of this plane has to be 
replaced by a curved surface. 

In Figure 5.4.1 the new paths appear as AF and others below it, such as 
ED. Below F, 7; is to be replaced by the section shown dashed, where 
Sı changes sign. The full picture is seen in Figure 5.4.2. The “seam,” 
where the new curved surface replaces the old J, is found by considering 
paths emanating from AB. 

What means this modified 7,? 

Broadly, the solution we have found tells us that, for T reasonably large, 
both players begin the partie with all their forces devoted to the long- 
Tange objective of attrition. Then, at a certain critical time for each 
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Figure 5.4.2 


before termination, they switch to all-out attack. These times essentially 
depended on the c, only; the forces z4, x, and manufacturing rates m,, m, 
do not matter. The curved part of 7, is an exception. It means that P does 
best by switching from attrition to attack earlier, at a time depending on 
x, and x. We seem to have a mathematical confirmation of the reasoning 
by P: “Ifthe later part of my attack effort is going to be nullified through 
my forces being annihilated, I had better compensate by starting it 
sooner.” 

There is a similar phenomenon on @», where x, can become temporarily 
zero. It occurs in the hatched area on the vertical “wall” of Figure 5.4.2. 
Paths emanate from here into & but do not affect the TS. 

Finally, note that the formulas given for V do not hold in the domains 
covered by the paths emanating from either @;. 


Exercise 5.4.1. Find the equations of the curved part of 7. In particular, 
show that, as suggested by the figure, sections of constant T are straight 
lines. 


Exercise 5.4.2. Analyze the situation regarding the paths emanating from 
Co 


Exercise 5.4.3. For both new classes of paths, find V and compare it to the 
old. 
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5,5. THE ISOTROPIC ROCKET PURSUIT GAME 


The pursuer P is driven by a thrust of fixed magnitude F,° but whose 
direction he can control; in this way he navigates. The evader Ẹ has 
simple motion with the fixed speed w. The action takes place in the plane 
and the payoff is the time to capture. 

Of course, without a gain of difficulty in principle, we could assign more 
complex and realistic kinematics to Æ, but in practice we would be en- 
cumbered with more state variables, more elaborate KE and a much 
more tedious analysis. If the kinematics in the first paragraph were re- 
tained, but the roles of P and E reversed, we would have a problem of 
about the same difficulty as, and in many ways analogous to, the present 
one. If we permitted intermediate navigational magnitudes, that is, 
allowed E speeds < w and P thrusts < F, nothing would be gained; for 
in optimal play (the reader can verify this) both players would employ 
their extreme values at all times. 

This problem is similar in broad outline to the homicidal chauffeur 
game. But the solution to the latter, although the local motion is usually 
very simple, entails numerous singular surfaces and will have to await more 
advanced theory. Here the differential aspects of the motion are more 
intricate but the solution is much more analytic; a single integration of the 
RPE suffices for an entire optimal path. Formally one might say that here 
we have no linear vectograms; all max and min are interior; there are no 
abrupt changes to and from extreme values which can be subtly taxing. 
Nevertheless we will find here a counterpart of the swerve maneuver 
(Section 1.5). 

We will also burden P with a friction drag taken as negatively pro- 
portional to his velocity. There were two reasons. One was that without 
the drag there is no bound on P’s speed and, although not too much 
realism is claimed for this partially idealized problem, completely uncurbed 
speeds seem too shocking a transgression. If the friction force is —k times 
the speed, there is, as is well known, a natural limit to the latter equal to 
F/k. It is the speed P would come to asymptotically if his thrust propelled 
him along a straight line. The second reason is the interesting question 
of what circumstances enable P to capture at all if w > F/k. Although 
such games of kind come later (Chapters 8 and 9), we will be able to infer 
the answer here. 

If the reader prefers to elide this complication of friction, he can do so 
by imagining the k in our analysis below always replaced by 0. We shall 


° More accurately F is the thrust per unit mass of P or specific thrust. 


eat e 
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indicate the places where such engenders a formal deviation of the analysis 
by square brackets. 

There is to be a capture radius / > 0, that is, capture is defined as 
occurring when |PE| < /. 

We are going to begin our analysis in the realistic space which has six 
natural dimensions. Later we will shift to a reduced & with n = 3, which 
is minimal. However, the former space makes the analysis easier (if 
lengthier to write) and its interpretation more transparent. 

To describe a state of P we need to know both his location and (vectorial) 
velocity, requiring four coordinates. Two, those of location, suffice for 
E. In the standard Cartesian plane let x, y be the coordinates of P and 
u, v the components of his velocity. The coordinates of E will be xz and 
Yn. The control variable of the former player will be ¢, the angle his 
thrust vector makes with the y-axis. For E, y will be the inclination of his 
travel direction to this axis. 

The KE are then ; 

=u 

j=v 

ú = F sin $ — ku 
ù = Fcos ¢ — kv 
tp = w sin Y 
Yn = W Cos p. 

The first two merely say that the rate of change of P’s affix is his velocity. 
The next pair equate his acceleration to the propelling specific thrust less 
the drag. The last two relate to E’s simple motion. 

As time of capture is the payoff, G = 1. 

We shall write the partial of V with respect to xg as V,, and similarly 
for yz. We have then for the ME,, 


uV, + vV, + min F(V, sin ¢ + V, cos $) — k(uV, + vV,) 
$ 
+ max w(V,7 sin y + V,m cos y) + 1 = 0. 
Y 


Putting p= VV2 + V2 


and Pa = VV x? + Vur 


we have, from Lemma 2.8.1, 


> (SX 


iso. cos $ = — 
P 


dipe ge 
PE PE 
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Thus we obtain for the ME, 
uV, + vV, — Fp — k(uV,, + oV,) + wWpg +1 =0. 


io) 


Our standard procedure leads to the RPE: 


r= —u, V,=0 
y = =v, V,=0 
o y. o 
u = F— + ku, Va = Va — kV, 
P 
y. o 
b= F2 + ko, = V, — ky, 
EL a Vn = 0 
Pr 
o V, 
Yg = -w E, Vin = 0 
Pr 


Now let us turn our attention to Y. It is the surface in & characterized 
by |PE| = l. We depict it by five parameters 


X= Sy 
Y = Sg 
u = s3 
v= 


tg = S + Ísin s5 
Yq = Sa + [COS 55. 


Here the coordinates germane to P are labeled freely, but E is picked a 
distance / away from (5,, $2) so that the vector EP has an inclination with 
the vertical equal to sz. 

To find the useable part, put r = |PE| so that 


r? = (£g — £}? + (Yg —y)*. 
Then for the points of @ 
lf = (l sin s,)(w sin y — u) + (l cos 5,)(w cos y — v). 
The useable part, being specified by 


max? <0 
Y 


Rate 
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is here those points of @ for which 


MAINLY EXAMPLES 


W — Ss, SIN Sg — S4 COS 5, < 0. 


Exercise 5.5.1. With a vector diagram obtain this result geometrically, } 

If the capture region is thought of as a disk centered at P, interpret the $ 
useable realistically (P can only capture with the forward portion, de. 
pending on his speed, etc.) : 


Let us complete the initial conditions by finding the V, on @. As here, | 
clearly V(= H) = 0, we have 


Y es Vey 
Os, 

Va = 0 = V, + Vye 
V,, =O= V, 
te 


V,, = 0 = (V, COS 85 — Vyg sin ss). 


From the last equation and then the first two, for some A, 
a 


—V, = Vin = Asins, (5.52) 


—V, = Vg = À COS 5s. 


If we now substitute these V, into the ME, we will have an equation 
for A. There will be two solutions corresponding to contact with @ (if 
considered as a circle about P) from inside and outside. Here we are’ 


interested only in the latter. 
Let us take s; = 0. For x on @, then we will have E a distance / directly: 


above P. Move E up slightly (increase yg); then V will become positive, : 
Therefore V,m > 0 when s; =0. The last equation above shows that: 


A> 0. 

Substituting now into the ME, yields 
—A(sq sin ss + s4 cos s) + wA+1=0 

(Observe that on %, p = 0, but pg = |AI; 


we needed to know sgn 2.) 
Thus 


it was because of the latter that . 


1 
Sa Sin S + S4 COS S5 — W 


which is positive on the useable part by (5.5.1). 
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We are now ready to integrate. Treating the V, first we at once see 
that (5.5.2) holds along the optimal paths (not merely on @) because from 


the RPE the relevant V, are all0. The third and fourth right RPE integrate 








to 
V, = —A(sin s;) L= 
a i 
V, = —A(cos s;) foe 
[If k = 0, these become V, = —Ar sin ss, V, = —Ar cos s;.] 


At this point we can profitably return to the optimal strategies. Noting 


that 


— aok 
Pe E 





p=A 


Px =A 
we find that 
sin $ = sins, cos $ = cos sy 
sin P = sin 55, 
or $= P=5. 

Here is our first significant result. Under optimal play both control 
variables will be constant and equal, that is, E flies a straight line and P 
maintains his thrust vector in a constant direction. Further, this 
direction is the same as that of E’spath. The final equalities state that at 
capture P will be directly behind Æ (in his flight direction sense). 


COS P = COS 5; 


Problem 5.5.1. How much of this result depends on the KE and how 
much on @? How would it be changed with a different type of termination 
(say a less regular capture region than a circle)? See the missile trajectory 
problem in the Appendix for such a discussion of a related problem. 


The quantitative problem of the optimal strategies is now reduced to 
finding the above mutual direction. It will of course be a function defined 
over &, 

We now integrate the left RPE. The results are, as the reader may 
easily check, 
er — —1— kr 


k? 





5) + F(sin s) = 


z=s5-s( 





u = sze" — F(sin s;) - (5.5.3) 


£g = Ss, + (l — wr)sin s; 


1 With no friction (k = 0), P’s path then lies on a parabola. 


atte 
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and y, v, Yg have similar expressions except that sin s, is replaced by 
COS Sz, and Sis Sz by Sos S4- 
[If k = 0, the first two expressions become 

£ == Sı — Sst + 4F7* sin 5, 

u = s — Fr sin s5]. 

The next step in a full formal solution is to solve these six equations of 

the state variables for the six unknowns, Si, ..- , 55» T. In particular, the 
resulting 7 (£, Y, . -> Yp) or t(x) will be V. 





If we define 
r o i +k 
Q) =l—wr+F Coc (5.5.4) 
a short calculation shows that 
1—e*" : 
tu — % — a u = Q(r) sin s, (5.5.5) 
and similarly 
E 
Yg— y- 7 v = Q(T) cos S5 (5.5.6) 


[For k = 0, the above becomes 
Ql) = l — wr + Fr 
čp — 2 — ur = Q sin Ss, etc.] 
We can now eliminate s; by squaring and adding (5.5.5) and (5.5.6). 


To do so leads us naturally to the type of coordinates that we might have 
used in a reduced &. 

Let r be the vector (£g — %, Yg — Y), the displacement of E relative to 
P, and u, the velocity vector (u, v). Then when we square and add, we 
obtain 





r? — Xr- u)” (==) ma a(t ==") = Qn? 657 


an equation that is to be solved for 7 = V. 
[For k = 0, (5.5.7) is 
r? — 2(r' u)r + ur? = Q2] 
Observe that Q is a fixed function for a definite game; it does not 
depend on the state variables-but entails only parameters. A plot of Q for 
F=3, w=2, l=1, k=1 (5.5.8) 


is given in Figure 5.5.1. 


11 A scalar (inner, dot) product. 
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3 
2 
Q 
1 
0 0 1 2 3 4 5 
T 
Figure 5.5.1 


The left side of (5.5.7) entails all the state variables that would appear in 
proper reduced &, as we shall soon demonstrate explicitly. 

Let us make a tentative assumption whose significance will be discussed 
later: 


Q(r)>0 forallr >0. (5.5.9) 
From (5.5.4) we see that 
Q(7) ~ E- w)» TO (5.5.10) 


so that our assumption (5.5.9) requires that F/k > w? or P’s limiting 
speed exceeds that of E. 

Note that the left side of (5.5.7) is bounded for 7 > 0. Further, it is 
always positive. For, regarded as a quadratic, its discriminant is negative 
by the Schwarz inequality. 

We see that for any interior point of @, (5.5.7) will be satisfied for some 
positive 7. For when 7 = 0, 


left side = r? > /? = Q(0} = right side 


and, for large 7, this inequality, as the preceding lines show, will be 
reversed. Let To = ToX) = 79(2, Y, -- - » Yg) be the smallest such solution. 
[For k = 0, (5.5.7) is an algebraic equation of degree four. As above, 
inequalities for large and small 7, show the existence of a positive root.] 

This step of solving the transcendental [algebraic] equation (5.5.7) 
essentially completes the computational solution of the problem. For 


12 Let us spare ourselves the coincidental case of Fik = w. 
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Figure 5.5.2 


79(X) itself will be V(x). Putting 79(x) for 7 in (5.5.5) and (5.5.6) enables us 
to find a unique function, s,(x) defined throughout &. The optimal 


strategies are 
$ = P = sx) (5.5.11) 


We shall leave much of the formal verification to the reader. If a partie 
starts from x and the players use the strategies (5.5.11), holding them 
constant during play, he can verify that the function 79, calculated at each 
succeeding position, decreases at a unit rate. After a time To, then, To will 
be zero (this is why we must use the smallest root) and r will be Zand be so 
for the first time. 

But it is much more enlightening to translate into the tongue of a 
reduced space. The new & will have three dimensions, and we can envisage 
the whole action clearly. 

Let P and E be as shown in Figure 5.5.2a with the arrow denoting P’s 
velocity, which has magnitude v.!° The relative coordinates, X and Y, 
shown for E are measured parallel and normal to this vector. We take 
X, Y, v as coordinates in our reduced space. In it @ appears as a cylinder 
of radius / centered on the v-axis (Equation: X? + Y? = /*), and é is 
defined as the half-space (X, F, v with v > 0) deprived of the interior of this 
cylinder as shown at (b) of the figure. 

To translate (5.5.7) into these terms we observe that 


r= X? + y2 
r:-u=vY 
w =v? 


18 Not to be confused with the old v, which we shall not use again. 
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so that (5.5.7) now appears as 


X? + [r- (=) Q%(r). (5.5.12) 


for X? + (Y — vr)? = Q? when k = 0.] 

We shall now examine the surfaces (5.5.12) in & of constant 7. If, for 
the moment, we also fix v, (5.5.12) is the equation of a circle with center 
at X = 0, Y = v (1 — e~*)/kand radius Q. Thus the surface we seek is a 
cylinder of this “radius” whose vertical sections are circles and whose 
axis is the line 

zan yad 
= U, = (==), v=A, 0O<å< œ. 
When 7 = 0, note that this cylinder becomes @. 

Figure 5.5.3 is a rough sketch of how the family of cylinders begins, 
that is, with + small. Figure 5.5.4 is a carefully drawn cross section at 
v = 2.5 for the particular data (5.5.8). 

Observe how the circles of constant 7(= V) for low values have a 
prominently visible envelope. The two curves of this envelope meet € at 
the boundary of the useable part. Our rejection of all but the smallest 
solution of (5.5.7) is interpreted here as the discarding of all the circles 
(for small V) except their upper arcs which span the contacts with the 
envelope. Thus between the envelope components we see a smooth 
family of curves of constant V (so labeled in the figure) which merge with 
the useable part of @, which of course is on the locus of V = 0. 

At about V = 2, we see that the envelope terminates. The circles 
burgeon from here on, and we only delete those arcs which would cover 





Figure 5.5.3 
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territory already labeled with smaller V. For all sufficiently large V (not 
shown) we retain the full circle. 

The envelope is a section of a surface in two parts in & of a species that 
jn Chapters 8 and 9 will be called barriers. Here the barrier is tangent to € 
at the boundary of the useable part. It is a surface that is never crossed 
during optimal play and marks discontinuities both in V and the optimal 
strategies. It delineates those starting positions leading to an optimal 
play which is a simple, direct chase from those involving a what we might, 
analogously to the homicidal chauffeur game, call a swerve. 
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If the starting position is a point nestled between the barriers, such as 4 
in Figure 5.5.4, the resulting play will be of the direct type; E will travel 


away from P with the latter dogging his heels. In this reduced space, for ` 


starting positions such as B outside the barrier, the resulting path in & will 
begin by first receding from @, then skirting the barriers, and reaching ¢ 


via the alley between them. “Physically” this means that the kinematics | i 3 


of P render him undeft enough to catch E directly; P cannot veer enough 
from his course to thwart E’s sidestepping. Hence P must at first lower 


his speed so as to make a sufficiently sharp turn and then go after E, who 


has by now fled to a position roughly rear of P’s starting point. 

The realistic behavior of P and E are shown for these cases in Figure 
5.5.5, (a) and (b). Here Sp, Sg denote the starting points of the players 
and Cp, Cy their positions at capture. 

At b, E “‘swerves” to P’s rear, penalizing the latter by forcing the sharp 


turn. P, an optimal player, anticipates the ploy; he points his thrust @§ 
accordingly in a concordant direction (which is that of E’s path). Even =F 
though the starting position at a is only slightly different, E could not get 2% 
away with the same maneuver here; he would be caught abruptly if he 3 F 


headed tailward. Note that V(A) = 0.8, while V(B) = 3.0. 


We can see now some significance in the assumption (5.5.9) that Q > 0. -f 
Were this not so, the “radius” of the cylinders of constant V would some- = 
where shrink to zero and the two barrier surfaces would intersect. We = 
shall anticipate Section 9.3, where this question is more fully discussed, by S 
remarking that, if the barrier is cut off beyond the intersection, the re- = 
maining surface appears to bound with @ a portion of &. For starting Z 


points within this portion the preceding analysis is valid. For external 
starting points, if E plays properly, P cannot capture at all. 

Thus, as was remarked after (5.5.10), if P is to be able to capture from 
all starting positions, F/k > w is a necesary condition. It is not sufficient, 
however. (See Exercise 9.3.1 or (9.3.7) if k = 0.) 


5.6 AN OPTIMAL PROGRAM OF STEEL PRODUCTION 


Here we demonstrate, by a simplified example, how our methods can be 
applied to certain production programs so as to attain the maximal yield. 
To fit our conceptions, the situation is best supposed “differential”; the 
discrete steps of actuality must be smoothed into continuity. What we 
obtain is a long-term view or a rounded-out picture. 

A nation or other large enterprise embarks on a program of steel pro- 
duction. We will suppose, as is actually reasonable practice, that a certain 


amount of extant steel is to be an ingredient for the manufacture of | 


additional steel. At any time the current supply of steel is to be allocated 
between this use, the manufacture of more steel mills, or the stockpile. 
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Our desideratum is to maximize the latter category at the elapse of a 
specified time T. How should we proceed? We might conjecture that at 
the outset, for example, all effort should be devoted to augmenting the 
number of mills; when this number is great enough we proceed at high 
blast to produce. When should the transition occur? 

At any time ¢ let M be the number of mills existing and S be the 
current supply of steel. Let yy, be the fraction of steel devoted to building 
more mills, pg the fraction devoted to making more steel so that 


Ym > 0, Ys > 0, Yu tvs <1. 


Since y pS is the amount of steel allotted to mill building, we can write, 
supposing a linear relationship, 
M = cy mS 
for some c > 0. In a unit time let the quantity of new steel that can be 
made from a unit of the old be a > 1. Then the rate at which new steel is 
produced is aygS, but steel is consumed by doing so at the rate pgS. 


Besides the steel allotted to mill building is an irrevocable deduction from 
the current supply and its rate is yS. Thus 


S = apgS — ygS — Yus. 


A third state variable is T, the time until the end of the program. 
Therefore we have for the 


KE: M = cyyS. 
$ = S[(a — l)ys — Ym] 
T=-1. 


The number of exant steel mills sets a bound to the output of new steel 
production. Therefore, for some b > 0, we must operate subject to the 
constraint - 

apgS < bM 


or ys <—. (5.6.1) 


Thus in this problem the control variables will not always be subject to 
constant bounds. 

For & we have the octant of 3-space: M > 0, S > 0, T > 0; @ is the 
surface where T = 0, and we parameterize it by the partial initial conditions: 


M=s>0 
S=s, 20 
T=0. 
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hese numbers, for example, at 4, yy = 1 — bM/aS, pg = bM/aS. Of 


The payoff is terminal, for our goal is to maximize S when T =§ í 
k course, 1, 2, 3, apply in &, and 1, 2, 4, 5, in &. 


Therefore 


H = Sp. E When we derive the RPE, the nonconstant bounds on y necessitate a 
The constraint (5.6.1) is effective only when bM < aS. If we defif ~ modification of our standard procedure. The change is simple; when 
b differentiating the ME with respect to a control variable, we include its 
R=S-— £) M appearance as an argument of Pm or Pg. Inasmuch as the ME, will 
, AEN À always be one of a L FQ j=l,...,5 (5.6.2) 
we can say, only when R > 0. Accordingly we divide é into the two parti he RPE are 
&, where R < 0, and & where R > 0, by the quarter-plane Z on whic the . i n 
R= 0. In 4, there are ample extant mills to handle all the steel, but if M = —cSPy Vm = ĝm,/ðM 
&, the upper limit on production rather than a supply shortage is incurre S = —S[(a — Deg — Pul, Vg = 0m,/0S 
by a mill. Figure 5.6.la depicts @; note that the vectograms have tg T=1 Vr =0. 
S uy 







ms = SVg(a — 1) 
ma = cVy (S —È M) + V; (bM ~ S) 


-1 
ms = Vs = bM 


mo = cS Wp — SVs 


Figure 5.6.1 


Figure 5.6.2 


angular headplanes in &, and trapezoidal ones in @. Such are engende 
by the constraints on the control variables which limit them to the shad 
regions shown at (b). 

The ME, is 


max {cSVyyy + SVsl(a — Dys — Yul} — Vr = 0. 
y 


We make some general observations. 


The value of the maximizing m, is constant over all of any optimal 
path. (5.6.3) 


For when V is smooth, (5.6.2) and the final KE show that yn; = 0. Ata 
transition surface one m, will supplant another, but they must have a 
common value at the junction. At a crossing of Z, m3, m,, m; coalesce 
and so are numerically equal. 


Al 
ways Ve>1, Vy>0. (5.6.4) 


To find @ we observe that, in either £} or &, we are maximizing a linea 
function of yy, Yg Over a convex polygon. The maximum will alway 
occur at a vertex. In Figure 5.6.2 we have written the value of the brace. 
the ME, at the appropriate places. The vertices will be denoted by th 
numerals encirlced in the previous figure, and the corresponding values} 
the brace by m,(j = 1,..., 5). We will also refer to the local strategies} 


For suppose, starting from a point (M, S, T) of &, we play optimally. 
If the steel were initially augmented by an amount S}, we could ignore the 
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increment and use the former strategy. Thus we could get a payoff at Consequently, 
least equal to the old Value + S,. The second inequality follows similarly 
if we ignore an increment of M. We can certainly use the old Gy, Gg, the pa 


constraint (5.6.1) being now the weaker. 


Q=ckr—1 
R = s, — )s — skr. 
a 









































Never is m optimal." (5-6.5) When T = 7, = S,/s,k — bjak, R vanishes; the path leaves £}. When 
For, from (5.6.4), it is always dominated by m; or ms. 7 = T = I/ck, Q vanishes; we should expect, by (5.6.6), the dominance 
If we put -cy y of m; to be yielded to m,. On any path one or the other occurs, depending 
h Q = Vu — Vs a on the relative magnitudes of 7, and 7, which depend on s,/s,. It is not 
taen m, — m; = OR (5.6.6) 3 hard to show that the two classes of paths together fill all of & with 
and m = OS (5.6.7) ; T < 72 and intersect all of points of Æ subject to this inequality. The 
3 surface, which we shall soon see to be a transition one, consisting of those 
The m, of Figure 5.6.2 and a simple computation give these results, ? points of Ê, for which 
The initial conditions are completed in the usual way; on & T=7,= o lri (5.6.8) 
oH oH ee , bela — 1) 
Vy =— =0, Vy=— = similarly is met at all points. 
ds OS, a It is clear that m, is out of the running, for from (5.6.7), m, < 0 as long 
We turn now to the solution in &. Ason@, Q = —1, R > 0, we have, + as S > Oand tr < Tz 
from (5.6.6) > : Now let us turn to paths on the far side of the TS. As initial conditions, 
and from (5.6.7) ne we use (the s; are fresh parameters) 
m <0. ; M =s, Vie = kr, =* 
As m, > 0, the max m; at @ is m,, that is, Pg = bM/aS, Py = 0. All ` e 
steel and no mill production appears reasonable near the end of the S = Sa, Vg=l 
program. T = 7, Vr = ks, 
The RPE h ; 
j E S Š 2 Taking 4 for strategies, we now have the RPE: 
M=0, Pu = kVz (= 2) 
; aM M = —es(1 — *™) = er V =(~£y + vs) 
S=-kM, Vy=0 E s) Aa a TA 
T=1, Vr =0 S=-bM+S Vs = Vy — Vy =Q 
where we have written, for short, T = Vp = 
—1 
| k=bf ae We shall obtain full results pertaining to the optimal strategies without 
| The integral integrating these equations, although to find V we may have to do so. 
| S ERES RES As long as a path remains in &, R must be positive. We shall show that 
| M=s, Vu = kr Q, which equals zero for 7 = 0, is otherwise positive. From (5.6.6), such 
ill impl th s 
| S = s — ks,r, Vs = will imply that m, > mg. 
We have 
T = = = . 
T Fr Ms ks, 2 P 
§ = bo(—£ Vx ¥s) — 0 
14 Tt will turn out that neither is m, ever optimal. Is there a way of establishing this a 
fact at this stage? = —k,Q + ckVg 
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where k, = 1 + bcja > 0. The elementary formula for the integral of such 3 
a differential equation, with Q(0) = 0, gives us 


Q= cker* | ey (u) du 
0 
and so, from (5.6.4), our result follows, as exponentials are always positive, # j 
We now show that m, cannot dominate; it follows that m,’s sway is q : 
permanent. From Figure 5.6.2 and the RPE F 
nai NEMS My an 
a 
In &, M > 0 and on the TS, as a > 1, 
Py = 6(-441) >0. 
a 


We assert that Var remains positive. If it did not, let 7) be the lowest + H 
for which Vu = 0. From the RPE, at 7 = To, tj 


Vai = b(—£ ha + V,) = dV, = 0 >0. 
a 


But it is absurd that the derivative of Y m be positive at its lowest zero, 3 

Finally we shall show that this class of optimal paths completely fills the ¥ 
part of &, beyond the transition surface. Let (M, S, T) be a point of this : : 
set, so that R = S — (b/a)M > 0. The progressive path from this point § 
will satisfy (reverse the signs in the RPE) 


M=cR 
S=bM—S 
T=-1 


and so 


R=bM—S— ČER. 
a 


pail SNE RS 






If at some later time R equals zero, let fẹ be the first such point. At t, 


bM = aS and 
R=bM—S=(a—1)S>0. 


and we have the same type of absurdity as before. Let us note in passing, 
had we started from a point of R (with strategy 4), this same reasoning 7 
shows that R > 0 and so the path would (progressively) enter &. i 

Thus R > 0. As M > 0, also M > Oand S = R + (b/a)M > 0. Thus 
the path remains in £}. Ultimately it must reach the transition surface and = 


eile ioctl ital 


nih 


a 
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so be the reverse of one of our retrograde paths. Thus Ëa beyond the TS 
is filled; besides a path goes through each point of 2. 
Finally, we treat &,. We shall show that everywhere here 


m, > m (5.6.9) 


and, in view of (5.6.5), the optimal strategy is fixed at 3. 
On @, from Figure 5.6.2 and the initial conditions, 


m = S,(a — 1), M} = —Se 


and so (5.6.9) holds. It also does on & for each point there is met by a path 
from &, on which m, or m; dominates and these agree with m, on &. 

We now show that if (5.6.9) holds at the initial point of a path, it does 
so throughout. The RPE here are (Hg = 1, Pm = 0) 


M= , Vu =0 
$=-Sa@-1), Vg=Va—1) 
f= ’ Vp =0 


with the general integrals (the subscript 0 denotes an initial value) 
M=M 0 
S= Soe, 
Then, for T > 0 (or 7o) 
M = S(cV y — Vg) = Seer (Ves =, Vege») 
= Soc V moe ™™ — Vago) < myo. 


Vu = Vmo 


Pay A (5.6.10) 


As mą remains blow its initial value, and mą, by (5.6.3), remains at its, our 
result follows. 

Finally, it is easy to see that the retrograde paths of (5.6.10) (left), 
emanating from @ and Z, will fill £, completely. 

The complete optimal strategies are mapped in £ in Figure 5.6.3. In 
Ê, where there are more than enough mills, we should, as we have just 
learned, devote all steel to steelmaking. But one typical possibility is 
shown if we start at the point X of £. We make steel exclusively until we 
have enough to saturate the existing mills. Then X is at Aon Z. From A 
to B the mills are used at their full capacity and the excess ingredient steel 
is used to build more mills. At B, time a/bc(a — 1) from termination, we 
continue capacity manufacture but stop mill building and stockpile the 
excess. We so continue until C, which lies on @. 4 

As our description of the optimal strategy is complete, and of the paths 
at least qualitatively so, the Value can be computed by routine integrations. 
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Figure 5.6.3 


5,7. INTEGRAL CONSTRAINTS 


The contents of this section will apply to a problem in the Appendix but 
will not be used in the ensuing text itself. 

In the classical calculus of variations there are problems where the paths 
are restricted to those which render prescribed constant values to one or 
more given integrals. The relevant generalization here is 


[uc ¢, pdt =C (5.7.1) 


the integral extending over the path from the starting point to Y. We will 
deal with a single constraint of the type (5.7.1); the generalization to more 


is obvious. 
We introduce a new state variable and adjoin to the kinematic equations 


Sai = —L(x, $, y). (5.7.2) 


The new @ is the same as the old with z,,, = 0. Following our usual 
procedure we discard all starting points but those lying in the plane: 


„nı = C. The condition (5.7.1) must hold and our general technique 





[5.7] INTEGRAL CONSTRAINTS 125 


ensures the minimax with respect to ¢- and y-strategies under this con- 
straint. Often the above new @ will be deficient in dimension. We then 
can proceed in our usual way (Section 2.3): surround the terminal curve 
by neighborhood of radius ô, use the boundary as @ and take the limit as 
6— 0. 

But an alternative procedure seems more logical. The purpose of a 
é-neighboring surface is to supply initial values of the V; We can do so 


directly by taking Ye es ey (5.7.3) 
ntl = — ae 


where 4 is a parameter that can label the family of paths emanating from 
@ (which is of dimension n — 1 in an (n + 1)-space and so should normally 
not seat an n parameter family of paths). Together with the usual n — 1 
initial conditions (on @) 


V= UR, j=l,...,n—1 


and the ME,, we have n + 1 equations to be solved for Vy,..., V4, in 
terms of Si, -© s Sn- A. 
The only KE in the augmented set in which x„,, appears is (5.7.2). 
Hence one of the RPE will be : 
Vig = 0. (5.7.4) 


It follows that A will be constant over each optimal path, retaining the 
value it had on @. 

The advantage of all this is that there are significant instances when we 
are interested in the restriction (5.7.1) only superficially; what really 
concerns us is 


[uc $, p)dt < C. (5.7.5) 


In practical problems such happens when there is a resource (fuel, time, 
etc) £„+ı With a limit which one of the players must not exceed. Usually 
we can expect that optimal play demands that he exhaust all the resource 
allotted him; then a condition (5.7.1) suffices to express the constraint, 
although (5.7.5) is the literally true criterion. 

But there are cases where too much expenditure of the resource, even 
though allowed, is deleterious. Such is manifested from the point of view, 
say, of the minimizing player P, at a state (point of &) by 


ov 
OX ns 
holding there. In view of (5.7.3) and (5.7.4) this condition means that 
A<0 


>0 
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(a) 
Figure 5.7.1 


at the relevant point of £. Hence we attain the optimal expenditure of 
resource by the restriction on the initial conditions of 


A>0. (5.7.6) 


Of course, when the integral may > C, but not fall below, we work only 


with A < 0. ea 
Let us make this all clear by a simple yet general graphic illustration. 
With n = 2, take KE of the usual form ż = f, ġ = g, so that the ME, is 


Vf +V,E+G=0 (5.7.7) 


where the bars indicate that the usual $ and # appear as arguments. In 
Figure 5.7.1a, @, specified by x = X(s), y = Y(s) and on it V = H(s), 
and the optimal paths are shown, assuming the solution to have been 
attained in our customary way. The part of the plane above @ is @. 

Now let us consider the new game which arises when we adjoin the 


strict equality constraint (5.7.1). To the KE is added 
“= —L 
and the new ME, is now 
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The set fg defined by zx, y €old &, u = C, is comprised of starting 
points such that the constraint (5.7.1) holds. 

For initial conditions we now use first the values (5.7.9) of x, Y, u and the 
values of the V, obtained by solving simultaneously 

X"(s)V,, + Y'(9)V, = H, (as in the case of no constraint) 
Vi=-A 
and the new ME (5.7.8) with the values (5.7.9) as arguments in f £ etc, 

From the RPE—the old ones with ù = L and Ý, = 0 added—with the 
above initial conditions we obtain a two-parameter (s and 4) family of 
paths emanating from @. That is, from each point of @ there is a family 
of paths, one for each A. We may expect that the paths will meet &, in a 
subset & of it as indicated in (b) of the figure. 

Suppose we wish the solution of the constrained problem with a specified 
starting point x°, y°. Then we find the corresponding point (x°, y°, C) of 
o- If it lies in 2, then the path through it will be the optimal one for the 
augmented game; and the projection of this path on the plane: u = 0 
will be the optimal one in the original space. 

If the point (x°, y°, C) lies outside of 2, then there is no path, which 
means, in the original terms, that it is impossible to get from the starting 
point to @ without violating the constraint. In practice, the boundary of 
&, which delineates those impossible cases, may usually be identified as 
the limiting curve in @ where A = œ. 

Note that on starting points of the curve in £ o for which 4 = 0, optimal 


play is the same as in the original unconstrained game. For if V,, = A = 0, 
it is clear that formal calculations of the original game are unchanged 
when we adjoin the constraint. The paths of (b) in the figure for which 
4 = 0, when projected on the u = 0 plane, are the paths of (a). 


Now let us turn to the case of the one-sided constraint (5.7.5). We shall 


assume that, in an effective two-player game, the constraint integral is 
governed by one player alone. For otherwise we might expect such 
phenomena as the opponent striving to violate the constraint; for the 














game to retain sense, he should reap a bonus in payoff if he succeeds and 
such appears to require a basic reformulation of the original rules. 
Certainly our assumption holds in many cases derived from reality. For 
example, if one of the players controls a moving craft, the constraint 
integral might express a range or fuel limitation and it would be this player 
alone who regulates its expenditure. For definiteness we shall take P, the 
minimizing player, as enjoying this control. 
The admissible starting points are now the set Q defined 
(2, y)E g, O0cu<c 
for such and only such ensure that f L < C. 


Vi.f+V,E+G—-LV,=0 (5.7.8) 


and we should note the ¢ and Ẹ, functions now of x, y, V,, Vy, Vu are not 
necessarily the same as in (5.7.7) and thus neither are fg, G. a 

Figure 5.7.1b shows the enlarged &. Note that the new @ is in the 
u = 0 plane and is otherwise the same as the old: 


z= X(s) y= Y(s), u=0. (5.7.9) 


The new & consists of the points above the surface which is the u-trans- 
lation of @, thus of x, y, u such that (x, y) € old &, u > 0. 
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Figure 5.7.2 


Now Q will be cut by the surface Z on which À = 0, as shown in 
Figure 5.7.2a, which meets the plane where u = 0 at @ and meets &> ata 
curve we shall call Zo. We shall assume an orientation which, in our 
figure, reads: A < 0 below ¥ and 2 > 0 above Z." 

Suppose now we are assigned a starting point x°, y° in the original 
problem. In the constrained problem we are at liberty to use any starting 
point on the horizontal line through Q: x = æ, y =y’, 0 [u< cC. 
The optimal will be that one where V is least. Suppose first the point 
x, y? in &g lies below Zo. Then the horizontal line through it will meet 
F at a point K. As we know, from the KE, Vu = 0, and the initial 
condition, V, = —A, that V,, = dV/du = —A throughout Q, V will vary 
along the line as at (b) of the figure. For to the right of K, V,, > 0, at 
K, V, = 0, etc. Thus the minimum occurs at K; it should be the starting 
point under optimal play. As 


fear=uerK<c, 


the optimal path should be the same as if there were no constraint, and 
P may ignore it. 

On the other hand, if x°, y? (on ĉo) lies above Zo, then on the hori- 
zontal line, 


ALEE PE, 
ðu 


18 We shall not stop to investigate the validity or alternatives to this assumption. 
It would appear to hold in most realistic cases. 
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and the best starting point has u = C. For £o, Yọ on Lo, of course, best 
play is along the optimal path where A = 0 and the constraint if fulfilled 
exactly but without sacrifice. Thus we can conclude: 


In the augmented game, when À < 0 at the starting point, optimal play 
is the same as in the original unconstrained game but it must be altered 
when A> 0. 


Note that if the maximizing player controls the integral, this statement 
js still true if the adjoined KE is taken as V, = +4. 


Research Problem 6.3.1. What are the formal conditions that dictate that 
f L dt is under the control only of a particular player. 


For a pristine example we revert to the simple classical problems of the 
isoperimetric type. They concern curves of specified length bounding, or 
partially bounding, with diverse end conditions, the maximal area. The 
textbook standby has the curve joining two given points in the upper 
halfplane; the area beneath, in the sense of elementary calculus, is to be 
maximized. 


Example 5.7.1. A classical isoperimetric problem. To translate to our 
terms let the curve be described by a point moving with unit speed. 
Thus we write the 
KE 

z= cosy 

ý = sin y. 
As the area beneath is f ya dt we take 

G=ycosy. 


Because of the unit speed, arc length is the same as time and so to form 


(5.7.1) we take 
L=1. 
Thus the additional KE is 


= —l. 


We are led to the ME, 
max [(V, + y) cos y + V, sin y] — Vp = 0. 
Y 





Putting 
p=V(¥, +y +V? 
we have 
E A mpn 
P P 
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and the ME, is p—Vr=0 
RPE: 

ve 2 Vet) $o 

z= 2 
i V, + 

o — y, o. Ed y 
y= — y= 
o P o P 
T= Vr=0 


We can see at once that any integral path of this system is an arc ofa 
circle. For V, and Vp are constant; from the ME,, so is pe The linear 
subsystem of differential equations for y and V, have the familiar integrals 


y + V, = C, cos K + cı) 
P (5.7.10) 


V, = C sin (c + c). 
P 


Finally, we note from the RPE that x differs from — y, by a constant. 

To illustrate the initial conditions, let us take as the original @ the line 
Y where x = X > 0 and starting points with s < X, y > 0 T> x — z. 
We wish to find the curve of specified length T extending from this (x, y) 
to some point of Z which subtends the maximal area beneath it. 


The final @ is re X, y=s(>0), T=. 
On it V, (as V = 0 on LZ) = 0 = V, and 


Vo =A. 
In virtue of our discussion we limit ourselves to 
A> 0. 
Then also on @, using the ME,, 
p=Watsl=4 


As the nature of the paths dictate that £(0) < 0, the first RPE shows that 
on &, V, +y > 0. Then the initial conditions are completed by 


V,=A-—s. 
Tailoring the integrated RPE to fit these, we have 


a 
= X — Asin- 
j a 


=s—a(t — cos] 


T=r7. 


(5.7.11) 
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Clearly our curve is the arc of a circle with radius 2 and center on Z 
with ordinate s — A. Obviously if T is within reasonable limits, there will 
be just one such arc of length T, ending at (x, y) and meeting Y at a point 
above its center. This is the well-known classical answer. 

We make the obvious observation that from the ME, 


so that we never lose area because of too much length. 

If T is large enough, we should realize that there is nothing in our 
formulation, prohibiting the point from making more that one circuit of 
his circular optimal path. Area is of course reckoned with sign as in 
elementary calculus. 

Finally, let us study the limiting case as å — œ. It is easy to see that in 
the limit (5.7.11) becomes 


x=X—T 
=s 
T=7 


corresponding to starting points whose distance from @ is just T, so that 
there is no choice of paths. These cases delineate the possible. 


Exercise 5.7.1. Solve the typical classical isoperimetric problem where the 
sought curve is to connect two given endpoints in the upper half-plane. 
(Surround one of the endpoints, say P}, by a circle of small radius ô to 
obtain @, etc.) 


Problem 5.7.1. Make a two-person game of the last exercise by allowing a 
second player P to move P, with simple motion at a speed <1. His 
objective will be to minimize the area under the curve. Its length is still 
to be T, but P can be considered victorious if he can render such impossible 
to the original player E. 


It is hard to mine anything new from such well dug ground as these 
isoperimetric problems. But the problem below is novel in that it asks for 
fewer end conditions than its classic brethren. Yet it is meaningful and 
fits our ideas nicely. 


Problem 5.7.2. What curve (or curves) maximize the area beneath when 
the only constraints are 

1. The length is a given T > 0. 

2. One endpoint is given? 


Research Problem 5.7.1. Show that, up to an arbitrary multiplicative 
constant (perhaps + 1), our å is the same as the Lagrange multiplier in the 
classical treatment. 












































CHAPTER 6 


Efferent or Dispersal Surfaces 


The rich variety of singular surfaces which can be entailed in the 
solutions to differential games is the key to the important, often pre- 
dominant, phenomena which fall outside the scope of mere differential 
equations. This chapter is the first of several which emphasize a par- 
ticular type. Dispersal surfaces, although simple in principle, are often the 
seat of mixed strategies and conflicting decisions by the participants. 
There has been a great deal of misunderstanding about such situations, 
which we endeavor to clarify in the central sections. 

The first section contains a classification scheme for singular surfaces 
and some general observations on their roles. Dispersal surfaces are 
defined, singled out, and illustrated by examples. 

Section 6.7 is devoted to the geometric method of solving certain pursuit 
games. Here too are the above difficulties—special positions which at 
first appear to involve an unresolvable decision balance. 

The final sections contain new examples, some presented as problems. 


6.1. SINGULAR SURFACES 

As has been mentioned earlier, there are frequently two aspects or stages 
to the solution of a problem of differential games. One, termed the in the 
small, concerns the integration of the RPE, and in the examples of 
the previous chapters this procedure bulked large. The other, termed in 
the large, consists of ascertaining certain singular surfaces which gener- 
ally separate regions of different behaviors of the integrals of the RPE. 

A singular surface is an (n — 1)-dimensional manifold in é on which 
regular behavior of the solution, as an integral of the ME, fails. 


132 
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To classify singular surfaces, we can consider the optimal paths on the 
two sides, for we assume, at least locally, that a surface, interior to & 
separates it into two regions, locally “sides.” There are four obvious 
possibilities for the path behavior on each, which are, with their notations: 


The paths enter the surface. (+) 

They leave it. (—) 

They do neither, that is, are parallel to the surface when sufficiently 
nearby. (p) 

There are no paths. (0) 


As each condition may hold on either side there are 16 possibilities, 
which we designate by such symbols as (—, p). Further, the surface itself 
may be comprised of a set of paths; we will denote this event by (u). For 
example, the singular surface of Figure 6.1.1b is of type (+, u, +). 

This classification thus comprises 32 possibilities. Not all of them need 
be realized in practice, for example, the cul-de-sac (+, +). Transition 
surfaces, where a control variable abruptly changes in value, have appeared 
in examples of the preceding chapter. They are of type (+, —), but so is an 
ordinary surface in & if it cuts an optimal path at each of its points. (But an 
ordinary surface, if a union of paths, is (p, u, p).) 

This classification is exhaustive geometrically, that is, in terms of the 
configurations of optimal paths. As such it suffices for most of our 
purposes, but full consummation would demand in some cases a finer 
subdivision in terms of the local optimal strategies. For example, from 
each point of a singular surface of type (—, u, —) three paths emanate. 
Optimal play could require x to stay on the surface (the (u)) when there, 
but take a branch path (the proper (—)) when slightly away. Or all three 
choices are optimal. Or a mixed strategy (such is not impossible, as we 
shall shortly see) is best, etc. Further a (0) may signify that no solution 
exists or many do with all paths optimal. 

But mere taxonomy in itself does not constitute a theoretical structure. 
We have as yet discovered no unified theory based on the above classi- 
fication. On the contrary, the ideas surrounding the various kinds of 
singular surfaces seem extremely disparate. Much of the ensuing text will 
be devoted to particular types. 

Differential games appears to have an extraordinary furcate structure. 
No matter to what extent the theory is seemingly mastered, new and 
perplexing phenomena crop up, even in the most innocuous looking cases. 
We shall drop a sample of such in the reader’s lap in Section 6.10. 

As we successively master each novel quandary, some type of singular 
surface usually is the key. For they delineate, as boundaries, the regions of 
special phenomena or they act as the bearers of initial conditions which 
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(a) (b) 
Figure 6.1.1 


generate new families of paths. Thus it is that a theory of singular surfaces 
based on their types will be recondite, perhaps nonexistent, and it is thus 
that we devote so much effort to particular types. 

In this chapter we shall study the type (—, —), depicted, for n = 3, in 
Figure 6.l.la. At (b) is a specimen of type (+, u, +), such as will be 
studied in the next chapter, which with its tributary paths, may have the 
same geometric configuration. Despite their pictorial similarity, these two 
singular surfaces signify very distinct phenomena. 

We shall term the (—, —) case a dispersal or efferent surface, and the 
(+;u, +) a universal or afferent surface.1 As we shall soon see, the 
former entails a quandary: one or both players must decide by which of 
the two routes he shall leave the surface. On the other hand, a universal 
surface is an assembly of especially advantageous routes to the player con- 
trolling the creation of such a surface and the tributary paths generally 
represent his preliminary efforts to reach this desirable highway. 


6.2. DISPERSAL SURFACES 
A rudimentary but typical instance occurs in a game of tag with a large 
obstacle—say, circular—in the playing region. We assume the players 


1 The latter choices of names are of course borrowed from the neural nomenclature. 
However, the former choices are more descriptive of the roles of the surfaces and we 


prefer them. 
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Figure 6.2.1 


have simple motion, with P’s speed greater than that of E. If time of 
capture is the payoff, it requires no sophisticated analysis to realize that the 
course of an optimally played partie will often be as sketched Figure 6.2.1. 
The tangent to the circle (at b) through his initial position will be E’s best 
route, while P will travel first along the tangent Pa to a, then will traverse 
the arc ab until hot on £’s trail bE, remaining on this line until capture. 

But suppose that the starting positions both lie on a line through the 
center O of the circle with the latter between the players. The symmetry 
besets each player with a quandary; each has two equally good tangents to 
use as routes. The set of all such symmetric positions constitutes our 
simple but typical dispersal surface.? 

Another instance of a dispersal surface is in the homicidal chauffeur game. 
Should E be directly and sufficiently far rearward of P, each player will 
have two optimal strategies; P, for example, must choose between a 
sharpest left and right turn. 


6.3. THE NATURE OF DISPERSAL SURFACES 


We are assuming as part of the definition of a dispersal surface that 
strategies exist so that either of the two optimal paths emanating from any 
point of it, may be utilized. As a consequence, it is clear that the traversal 
of either of a pair of paths from the same point of the surface will lead to 
the same payoff. For this payoff will be the Value of the game starting 
from this point. 

In a one-player game, consequently, the player may choose either path 
indifferently, both being optimal. But with two players, there may be a 
characteristic dilemma: the choice of each player will depend on that of his 
Opponent. Generally one player will desire matching choices; the other 
the opposite, so that at the instant they are doing something like playing 


? Clearly a reduced space of three dimensions suffices: say, the distances OP and OE 
and the angle POE. The set where the latter is «+ must contain the dispersal surface. 
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Figure 6.3.1 


the elementary game with the matrix 
i -l 
-1 L (6.3.1) 


In the game of obstacle tag, for example, P and E may choose the optimal 
directions 1 or 2 of Figure 6.3.1. Here, say, E might choose 1, guessing 
that P will pick 1 and pursue by the upper route; E thus strives to match 
choice numbers. On the other hand, a mismatch is to P’s advantage. 

This dilemma of interdependent choices does not appear serious, for as 
soon as play progresses any positive amount one can expect that x will 
have moved off the dispersal surface. Thus we will resolve matters by 
what we term an instantaneous mixed strategy (IMS). Let some small 
positive e be chosen. Let the players make their path decisions with the 
optimal strategies pertaining to the matrix (6.3.1), that is, choose their 
alternatives each with probability 4. They persevere in these choices until 
the elapse of time e. The position being no longer on the dispersal surface, 
the partie proceeds in the ordinary way, aside from a possible small 
penalty accruing from a wrong guess. 

The indefiniteness of the “small” e is undesirable, of course, in a theor- 
etical problem of mathematical analysis. But in any practical application 
there will be a certain imprecision which will suggest a feasible value of e: 

the amount of time, for example, required to detect and act upon the 
opponent’s decision. 

However, if K-strategies are adopted, there is no difficulty at all. The 
first decision (at the dispersal surface) is mixed, the rest are pure. 

There can be dispersal surfaces which do not require an IMS. One 
player may be faced with a choice of optimal strategies when x is on the 
dispersal surface, but his opponent’s may be unique. Then clearly the 
former player may take his choice indifferently.® 


2 An instance occurs in the bomber and battery game in the Appendix. When the 


bomber approaches a straight coast from a point on its normal from the battery O, 
he battery’s firing strategy is the same 


he has two symmetrical, optimal routes. But as t 
for either, the bomber gains nothing by mixing. 
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6.4. THE QUESTION OF 
DEMNA THE PERPETUATED 


The effect of an instant i 
e efi aneous mixed strategy, bei ini 
: an i gy, being of m 
peated aes ia is not great. But there are cases oe the reed ‘ 
a ng k po appears to endure throughout an appreciable Han 
o such a phenomena as a } i 
) 1 e perpetuated dilemma. i 
is perhaps best clarified by illustration and the following is ne = 


Example 6.4.1. The wall i 
, 1. pursuit game. Here P and Eh i i 
va pene speeds w and 1 with w > 1.4 The playin ap ae eae 
a PI na a boundary W being the “wall” which E may Bot i We 
rested here only in starting positions with Eon W. C hele 
sets - Capture time is 
oe nee capture to mean the coincidence of P and E. 
eae : ef hes taken as obvious that E will do best by always remaini 
pt a Se pi wall W or we can adopt this limitation as a sonata 
eae a Meen a Poora of some intrinsic interest—the 
) ) constrained i 
ea ee sh elbes ed (or assumed) to motion along a 
From iti in Fi 
a ae Lota in Figure 6.4.1a, the optimal partie is clear: Æ will 
oo nd P will select the straight line path which will enable hi 
anal ptd ehi the point labeled C. Itis well known and easy to : m 
is “collision course” navigation b i i 
S y P, the straight li 
PAPE retains the same slope. (Hence the uae of Roni x 
& navigation” against evaders constrained to a constant ody 


Ww oA 
C 
1 
P 
2 
E w 
P C2 
(a) (b) 
Figure 6.4.1 


Ar bitr ar y speeds may be used with Z s the gr eater > wcan be construed as their ratio. 


a o 
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Our dilemma occurs when PE is perpendicular to W. Then both players 4 
are confronted with choices 1 and 2, ((b) of the figure) somewhat as held-4 
in obstacle tag. Supposing that play proceeds under congruous choices 4 
(such as 1 and 1), then the constant bearing principle shows that PE will 4 
remain perpendicular to W. Thus at each instant of play, the possibility is 4 
open to E of reversing his velocity without sacrificing optimality. The 
fact that P, always alerted for such a reversal, is similarly always faced 3 
with a corresponding pair of choices is our sought instance of the per- 4 


petuated dilemma. 


The reader can possibly conceive of other simple games sharing this, a 
phenomenon. How should the players act? To mix strategies con. $ 
tinuously throughout a positive interim seems theoretically absurd. But 4 


this difficulty is hardly grounds for jettisoning our science. 
In many of the cases based on reality the trouble is merely caused by the 


problems being improperly set. If the terminal manifold is of smaller | 
dimension than n — 1 (such is true of the example due to the stipulation `; 


of pointwise capture), the deficiency can be reflected as a perpetuated 
dilemma. Once the proper dimension is restored, the difficulty evaporates, 


The reason becomes apparent when the game is put into a suitable reduced ; 


space. We return to our illustration. 


Let us change to a proximity capture, that is, restore a @ of the proper ` 
dimension. We center P in the disk of radius / (Figure 6.4.2), the capture 
region. Again supposing perpendicularity at the outset, collision course -f 


navigation will carry E to some point C and P to P, (which lies on the 








Figure 6.4.2 
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x 


Figure 6.4.3 


segment CP such that |CP,| = /). Observe now, that progressively the 
segment PE changes its slope: it becomes less horizontal until at capture 
it becomes P,C. Thus once the original perpendicularity is broken, it never 
recurs. 
Despite the simplicity of this game, a formal analysis is enlightening. 
We shall require it in any case to illustrate the ideas of the next section. 
It is clear that we can employ a reduced space with two coordinates x, 
y, as in Figure 6.4.3a. 
The KE are obviously 


z= —wceos¢ 

ý =y — wsinġ 
with w>1 
and -l<y<l 


so that the ME, is 
mig max [—w(V, cos ¢ + V, sin $) + V,y] = —1. 
Y 


Thus if p = VV + V}, then cos $= V,/p, sin ¢ = V,/p, while 
y = o = sgn V, and the ME, becomes 


—wp + oV, = —1. 
The RPE follow: 
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Taking s(—7/2 < s < 7/2) as in (b) of the figure, @ is described 


æ = l cos 
y = lsins 
so that V, = 0 = (— V, sin s +V, cos s) : 


Hence ; 
z = À cos s, V, =Asins. 


As the Value gradient vector (Ve, V,) is to point into the space, clears 
A> 0. We do not require its specific value. Finally, 





o = sgn V, = sgn sin s = sgn s. . 
A g v` 8 8 : Figure 6.5.1 
As V, = 0, the V; retain their values above and we are to integra 7 
& a eee d As Is] < 7’ the first equation implies s} = —s, = s so that the second 


y = wsins — o oe = (/ i i 
y = (l + wr) sins — 7 = —[(/ + wr) sins — 7] 


which, with the initial conditions yield e a 
i implying that y = 0. Thus the x-axis (with æ > /) is a dispersal surface and 


the only such. The complete path map is sketched in Figure 6.5.2 
Recall that in our first version of this game, where capture meant the 
eoincidence of P and E, we were beset by a perpetuated dilemma. To see 
ow such arose, in Figure 6.5.2, let /, the radius of the capture region 
| become zero. From the figure it is clear that all paths such that 


x = (l + wr) cos s 


y = (l + wr) sin s — or, (is < E ,7T > 0, o = sgn s) (6.44 

which describe the optimal paths. We shall return to these equations k, 

the next section. § 

[sin s| < wt 

6.5. THE CONSTRUCTION OF DISPERSAL 
SURFACES 


Let us suppose we are at the stage of the solution process where we havl 
just integrated the RPE from some set of initial conditions as might vi 
borne, say, by @. Suppose we find that the paths obtained fall into twa 
classes which intersect (members of one class meet those of the other, nde 
of their own) such as is suggested by Figure 6.5.1. We find the locus ofa 
which two paths, one from each class, meet and such that the Value at thé 
meeting point is the same for both. This locus is the sought dispersal 
surface. Only the segments of the paths extending from the seat of theme 
initial conditions to the dispersal surface are retained. 

We illustrate with the preceding example. Returning to the equation 
(6.4.1), we see that the paths can be divided into two classes according af 
s Z 0. We seek the points where two paths, one from each class, meet witli 
the same Value which here is r. That is for any fixed 7 > 0 we must havé 


will then coincide with the dispersal surface and it becomes the renegade 


path. 
y 








(AB is paraliet to DS) 
x = (1 + wr) cos sı = (l + w7) cos s 
y = (l + wr) sins, — 7 = (l + wr) sins, +7 


where 5, >0> So Figure 6.5.2 
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6.6. FURTHER EXAMPLES 


In this and some later sections are further cases where the perpetuated 
dilemma is dispelled by use of an orthodox @, but the means are dis- 
similar. Simple as is the example below, it does typify a situation which 
might well be embodied in more significant games. 


Example 6.6.1. The one chance pursuit game (M. Dresher). Because this 
is a game of kind we will, to a slight degree, have to anticipate the techniques 
of Chapter 8. 

Figure 6.6.1a shows the vectograms of P and E which are the same for 
all positions. The desideratum of P is to attain capture (in the point 
coincidence sense); E to avoid it. It is clear that once P and E pass one 
another, the chance for capture is irrevocably lost. 

It is when the line EP has a slope of 45° that the perpetuated dilemma 
occurs. Then:P must continually outguess E in order to attain capture, 
and inversely for E to escape. 

Taking x and y as the relative coordinates of P with respect to E, we 
can use them as reduced coordinates, as shown at (b). The vectograms 
for both players are identical. Capture consists of bringing x to the 





(b) 
Figure 6.6.1 


5 We are taking the vectograms to be the same size, but only a slight modification is 
necessary if they are different but similar. 
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Figure 6.6.2 


origin. The dotted line is one of perpetuated dilemma, for if E moves 
vertically, P must do so horizontally to stay on it and vice versa. 

Let us see to what degree the contretemps is dispelled by employ- 
ing a capture region of positive size. In Figure 6.6.2 it is taken as a 
disk. 

From any point of the shaded region, between the lines B, and B,, P can 
achieve capture. These lines are what we will later call barriers, and the 
reader will have a better understanding of their role in games of kind after 
having read Chapter 8. Here it suffices to note that if x is in the shaded 
region, P can keep it there by playing vertically when x is sufficiently close 
to B, and horizontally should x approach too close to B,. This policy can 
easily be embodied into a definite strategy by assigning definite bounds to 
the closeness. 

Entirely analagous remarks apply to E’s behavior when x is outside the 
shaded region. He can prevent P from forcing the entry of x and so can 
achieve escape. 

Thus the outcome of the game is precisely determined at all points save 
those on B, and B, themselves. But on these lines the strategy of each 
player is uniquely determined under penalty of definitely losing the game. 
On B,, say, P must play his vertical velocity and E his horizontal, each having 
to act so as to keep x from shifting to the region unfavorable to him. 
Thus, once on B,, the point x will traverse it. The result is that it will 
touch the capture disk without entering it. 

Of course, we are at liberty to define such an outcome as either capture 
or escape. In Chapter 8 we shall give grounds for calling it neither, but 
terming it neutral. 

But at all events, the perpetuated dilemma has been dispelled; un- 
ambiguous strategies can be defined for all &. 
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Before proceeding with further examples, it is well to digress on 


6.7. THE GEOMETRIC METHOD FOR SIMPLE 
PURSUIT GAMES OF KIND 


The simple problem of the interceptor and bomber given in the first3 
chapter (Example 1.9.2) is an illustration of the ideas to be somewhatg 
expanded here. We deal with pursuit games 
and, for simplicity, ones with simple motion j in} 
the plane. q 

Let the set of points which E can ‘ead 
without being captured, despite P’s best 
efforts, be called the safe region and let the! 
surface which bounds this region be desig-4 
nated by the. BSR (boundary of safe region)/4 

In many instances of pursuit games it is clear that under optimal play4 
capture will occur on the point U of the BSR yielding the highest payoff 
to E. The optimal strategies will be such that both players will travel to. À 
U in minimal time and capture will occur there. } 

These concepts have all been illustrated in Example 1.9.2 where the | 
BSR is the perpendicular bisector of PE. 4 

We wish to note here a few particulars about various types of BSR, § 
If the ratio of E’s speed to that of P is w, and capture is defined as co- 4 
incidence of P and E, the BSR will be the set of points U, where i 


|EU| = w [PU]. (6.7.1); 

For w # 1, this set is the well-known Appolonius circle (Figure 6.7.1) 

If w < 1, E is interior to the circle and P exterior. Remark the readily 
proved fact: 

If P and E both travel straight toward a point U on the Appolonius 3 


circle, then any new such circle, obtained from a pair of simultaneous 
intermediate positions of P and E, is tangent to the original circle at U. 


(6.7.2) 


ye 


Figure 6.7.1 


If we replace coincidence by a circular capture region @ of radius /, the < 


BSR becomes the oval : 
|EU| = w[|PU] — 7]. (6.7.3) < 


If w = 1, the locus of U becomes a branch of an hyperbola. Some 
geometric properties, easily proved, are worth noting. 


€ But not in all. If Example 1.9.2 is modified only by assigning a lower speed to P, 
then Æ can always reach the target. 
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Figure 6.7.2 





The hyperbola passes through the midpoint of E and the nearest point of 
€ to E. The asymptotes pass through the midpoint of P and E and are 
perpendicular to the tangents from E to @. Furthermore, P and E are 
the foci (Figure 6.7.2). (6.7.4) 


Problem 6.7.1. Prove the assertion of the footnote: If P and E both have 
simple motion, but £ is faster, P cannot forestall by prior capture E’s 
reaching a given target. Capture means |PE| < /. 


6.8. FURTHER EXAMPLES: THE FOOTBALL PLAYERS 
AND THE COOPERATIVE CUTTERS 


Example 6.8.1. The football players. The vertical lines of Figure 6.8.1 
are the sidelines of a football field. The ball carrier E desires to move as 
far upward—toward his goal—as possible. He is opposed by the single 
tackler P. Both players have simple motion with the same speed. Capture 
is positional coincidence. 

The problem can be handled in the manner of Example 1.9.2: We draw 
the perpendicular bisector, the BSR, of PE and seek its uppermost point 
U in bounds. Both players run toward U, which generally lies on a 
sideline. 

Our dilemma occurs when the segment PE is vertical. The bisector now 
parallels the goal lines, and all its points are equally meritorious as 


destinations. There are not merely two but a continuum of equally good 
_ choices confronting both players. 


One is tempted to offer the solution: P should always remain at the 
mirror image E about the bisector. But this policy is not a strategy, for P 
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Figure 6.8.1 Figure 6.8.2 


would have to base his directional decision of E’s velocity, a control 3 4 
variable.” We have discussed earlier our motives for rejecting such pro- — 
cedures. a 

As before, we alter capture to the entry of E into a disk of radius] >f 
centered at P. As stated in the last section, the bisector is replaced by the ` 
arc and hyperbola. When PE is vertical, the quandary evaporates; E and P : 
should each run directly forward (Figure 6.8.2). : 

Note the degree of resolution here. We have passed from a situation of . 
infinitely many choices to one with no ambiguity at all! Even an in- 
stantaneous mixed strategy is not required. 

If E finds himself underneath P, within the vertical projection of the 
capture circle, the geometric analysis would indicate that the sidelines 
would generally play no part in the solution. For the result (6.7.4) shows 
that in this case, the two asymptotes would slope downward as they 
progress from the midpoint of PE out toward the sideline. If the side- 
lines are at an appreciable distance from P and E, the hyperbola 
branch will also slope thus, and its highpoint must be well within the 
ball field. 

The formal analysis below will confirm this statement. It leads to a very 
simple solution free of any reference to algebraic curves such as are the 
BSR. We dispense with the sidelines and, on the grounds of this slight 
increase in generality, assign a new title. 


7 However legitimate strategies exist which give arbitrarily close approximations. 
If P starts from a position near but not at the mirror image, he can adopt the strategy 
of always traveling toward it. 
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Example 6.8.2. The simple blocking game. The state and control variables 
are as in Figure 6.8.3 so that the KE are (the speeds being unity) 


jı = —cos $ 

Ya = cosy 

= —sin $ — sin y. 
We have a terminal payoff with H = yz. 


The ME, is 
mn max [—(V, cos $ + Vz sin $) + (V cos y — Vasin y)] = 0 
Y 
so that if Pı FNV V + V;2, P2 = VV2 + V2 


y, . V, = 
cos $ = —, sing = =; copa ingens 
Pı Pı Pa Pa 
Thus the ME, is 
è —pit p= 0. 
We have for € 


Yz = S t=lsins, y, = 5+] coss, 
(sı is angle of PE and the vertical.) As on ¢, V = s, 

Va = H, = 0 = (V; cos s, — V; sin s) 

Vo =l= ntn 


Thus ; 
V, = coss», V= Asins, V,=1—Acoss,. 


If s, = 0, so that capture has occurred with E at the lowermost point of @, 
then V, = 0V/dy, > 0 as we can see that increasing y, (with y, and x 
fixed) increases V. Thus A > 0. The ME, gives 


pı = A= p = V(1 — å cos s)? + Asin s, 


——!l 
2 cos 5, 


At this point we can note the optimal strategies and adduce a complete 
picture of a partie without integrating. 


from which 





V, Acoss, 


cos $ = — = A = cos 5, 
Pı A 
sin ¢ = sin s, 
- 1—Acoss 
ae S 1 = cos S, 


sin ~ = —sin sı 
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Figure 6.8.4 


so that 
$ = 54 = ý 


and the optimal paths must appear as in Figure 6.8.4. As the equal speeds 
imply |P,U| = |EU|, and as the two angles s, are equal, it follows that EP, 
is vertical. The simple construction for the optimal Strategies follows: 

From Æ draw a vertical line meeting @ at P, (the lowermost intersection), 
Draw PP, and extend it to U, where it meets the perpendicular bisector of 
EP,. Both players head toward U. 

Should E lie without the vertical projection of &, the second remark of 
(6.7.4) about the slope of the hyperbola’s asymptotes shows that by 
steering for a sufficiently remote point on this curve he can attain as large a 
payoff as he pleases. Thus no solution then exists. 

It is easy to see that this solution jibes with the geometric one. First U 
lies on the hyperbola from the definition of the latter as a locus. That the 
tangent is horizontal at U follows from the well-known property of a 
hyperbolic mirror’s reflecting a light ray emanating from one focus as if it 
came from the other focus. 


Exercise 6.8.1. Find the Value of the football problem when it is a 
composite of Examples 6.8.1 and 6.8.2. That is, the field has sidelines 
and there is a positive capture radius. As before, both P and E have simple 
motion with the same speed. The sought Value is the gain (of distance 
toward his goal) that E can make with both players acting optimally. 


Example 6.8.3. The two cutters and the fugitive ship. There are two 
pursuers (the cutters) P, and P, chasing a fugitive E. All move with simple 
motion, the speeds of the cutters each being greater than that of E. 

Time of capture is the payoff and coincidence of either P, or P, with 
E capture. 
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U 
P2 


Pi 
Figure 6.8.5 


The geometric solution proceeds as follows: Draw the two Appolonius 
circles (re P, and E, P, and E). The intersection of their disks (shaded in 
Figure 6.8.5) being the region in which E is safe from both pursuers, he 
heads for its most distant point U and so do P, and P}. 

Cieba® has given a rigorous proof, along lines quite different from ours, 
that this solution is correct. 

The perpetuated dilemma occurs when P,EP, lie on a straight line in that 
order. Then there are two points U both equidistant from E. Toward 
which should all three craft head? If they all pick the same one, they con- 
tinue during play to remain collinear and, by the remark (6.7.2), the points 
U are unchanged. We have optimal play but with a perpetuated dilemma. 

Cieba’s resolution is to have the pursuers take E’s velocity as well as 
position into account and is analogous to the mirror image policy of the 
tackler in last example and open to the same objections. 

But again we equip each pursuer with a positive capture radius. It is 
now clear that once the chase is under way, the three points lose their 
collinearity and the dilemma reduces to an IMS. 

Cieba points out that the symmetric case, with equally fast pursuers, 
with E midway between them, is equivalent to the wall pursuit game 
(Example 6.4.1). For if in this game we place a second pursuer, as a 
mirror image to the first, about the wall, the latter constraint may be 
removed, symmetry compelling E to follow his old course. Thus in this 
case of the two cutter game, at least, we have already done a full analysis. 


6.9. THE EXISTENCE OF THE PERPETUATED 
DILEMMA 


Can such existence occur in a legitimately formulated game? The 
answer is affirmative, as shown by 


Example 6.9.1. A game with perpetuated dilemma. For & we take the 
half-plane above the x-axis which shall be &. The payoff is to be terminal 


* We have read the ms. of his paper on pursuit games but do not know if and where it 
was published. 
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Figure 6.9.1 


with H(x) differentiable and even; it has a maximum when x = 0 and 
decreases as x— +00. The KE are 


i= +2Vice)+y, —-1S¢yS1 
y= 1. 


Thus x has a fixed downward velocity and must meet @; E desires it to do 
so as near x = 0 as he can; P, as far. The ME, is 


P + 2W/|xl)V, + PV, — V, =0 
where $ = —sgn V,, # = sgn V, and the RPE are simply 


2 e —ġ(l + 2y/jz)) = P, A acta ANI 
y= 1 = 0. 


o 


At € 
z=s, y=0, V= Hs) 


and symmetry enables us to work with s > 0 only. 
Then on @ 
V, = V, = H'(s) <0 


and so ¢ = 1, # = —1. From the relevant RPE it is clear that V, will not 
change sign and so these strategies persist. Integrating the system 


2 =—2Vzr, y= 
yields the paths 
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We complete the picture symmetrically in Figure 6.9.1. Clearly the 
upper y-axis is a dispersal surface. On it both players have the choice of 
any horizontal velocity of modulus <1. The maximizing player E will 
want x to remain on the DS and so will try to choose a velocity opposite to 
P’s selection. On the other hand, P strives for concordant choices. Thus 
as long as E succeeds in keeping x on the DS, there is a perpetuated dilemma. 

Even here we do not feel a serious conceptual hindrance. For suppose 
we quantize the game so that the players act in a discrete sequence of small, 
closely spaced moves, each a miniscule game with mixed optimal strategies; 
E loses once x moves off the DS, never to return. And the probability 
of each such loss is 3. Therefore it is extremely improbable that x stays on 
the DS for very long. If the continuous case is regarded as the limit of 
finer quantizations, it appears as valid to accept immediate removal. 
Certainly we may do so in practical problems. 


Problem 6.9.1. Show if the only change in the last example is to replace the 
first KE by 
è= + 2°) + y 


the paths are as in Figure 6.9.2. Thus it is possible to have a path of 
perpetuated dilemma that is not a dispersal surface. 

In Figure 6.9.2 there are no solutions outside the curves A. From such 
starting points P can obtain an arbitrarily small payoff. Thus the A are 
singular surfaces of type (0, u, p). 


It is easy to see that the paths near any path K of perpetuated dilemma 
must have the same direction as K. That is, if K is a DS the branch paths 
must meet in tangentially. This fact could have been used in our earlier 
examples to preclude the perpetuated dilemma directly, for there the paths 
were straight lines. 


x 


oO 
Figure 6.9.2 
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Figure 6.10.1 


6.10. VARIOUS PROBLEMS 


We mentioned earlier unsuspected intransigencies of differential games, 


As a case in point, we return to obstacle tag. This pursuit game was coined ` 


merely to introduce the subject of dispersal surfaces with as simple and 
obvious an example as possible. For such starting positions as sketched in 
our earlier figures, the solution is manifest enough not to require analysis, 
Yet for others the game is not so innocuous. 

Suppose Æ starts from very near the obstacle and P reasonably far away, 
as shown by E and P in Figure 6.10.1. If the players now act according to 
our former “obvious” optimal strategies, they will soon be at positions 
such as E,, P}. Here the obstacle no longer intervenes! The “obvious” 
solution is obviously incorrect, for the broken line is clearly a better path. 
None of the ideas in this book appears adequate to solve this game. 


Research Problem 6.10.1. What is the full solution to the game of obstacle 
tag? 

The problem below entails a dispersal surface determined from means 
other than mere symmetry. 


Problem 6.10.1. In the game with KE: 


& = ($ — ay)u(z) 
ý = —u(2), -l<¢y<l 
where 0 < a < 1 and u(z) is positive, has a minimum at x = 0, is otherwise 
monotonic, is infinite at x = + œ and j 1/u(x) dx exists. 

Here @ is y > 0 and @ is =s (0 <s < 0), y =0. The payoff is 
integral with G = 1. 


Prove: 1. There exists exactly one dispersal surface. 
2. It meets the x-axis at 0, lies within the sector 


|z| < (1 — ay 
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and is asymptotic to the line x = k, where k satisfies 


iN dz _ (de 
-o u(x) Je u(x) 


3. In the symmetric case where u(x) = u(—2) the above surface 
will be x = 0, y > 0, and on it V, is not zero and discontinuous. 


The following problem is difficult. As a military problem its ultility 
appears slight because of the miniscule advantages of the payoff, but it has 
some novel features, the understanding of which would advance the present 
theory. 

On the other hand, a game of kind version of this situation would be of 
immense practical importance if it is reasonably possible for the aircraft 
to dodge the missile by veering as described below. There would, of 
course, have to be some limitation on the missile’s turning capability, 
but he should navigate as best he can to anticipate the veering of his 


target. 


Research Problem 6.10.2. There are certain types of antiaircraft missiles 
which, due to physics of the means employed, can sense the proximity of 
their target much better from its rear than from its front. For example, in 
Figure 6.10.2a we will take the oval drawn about the airplane E as de- 
limiting the region in which the missile P can detect his quarry. In terms 
of our theory, then, the oval will correspond to the capture surface and we 
will denote its boundary by @. 

Let the airplane desire to advance as far as possible before capture, that is, 
P’s entry in @. In the position sketched, it seems plausible that E can 
achieve a gain by veering to the right when capture is imminent and so 
swinging @ away from P. Should @ be shaped as at (b), it appears that a 
similar gain would be witnessed by E’s veering left. 

Besides the usual aspects of a problem in differential games, this situation 
generates some unusual questions. 

Which way should the aircraft veer? [We have answered this: The 
veering is left or right (with P on the right as in the figure) depending on 
whether dD/ds > or < 0 at the point where P will meet @. Here D is the 
distance from Eto a point of @ and s the arc length of @ measured from the 
centerline. [Compare (a) and (b) of the figure).] 

It seems plausible that when P is far away, E flies straight. Just when 
does he commence to turn? 

The dispersal surface would be among the set of symmetric positions 
with P directly behind E’s tail. Is an IMS involved in one part of the DS 
(P and E close) and not on another (P and E remote)? 
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Figure 6.10.2 


Example 6.10.1. Sandpiles. If as much sand as possible is heaped upon a 
flat horizontal plate of arbitrary shape, the upper surface of the sandpile 
will have almost everywhere the same gradient, which is a characteristic 
constant of the sand. For example, a circular plate supports a cone and an 
oblong one, a hip roof. The crests formed by the five roof peaks are 
fairly obvious, but more irregular plates can yield some surprising con- 
figurations. 

Formally, if u(x, y) is the height of the pile, it must satisfy,’ except at 


crests, u? + u,2 = I (6.10.1) 


and u = 0 at the plate boundary, a closed curve @. 

Now consider the one-player, differential game: the point x, initially 
within @, moves with simple motion and unit speed and it is to reach ¢ 
in minimal time. We assert: 


The plot of V for the game is the upper surface of the sandpile and the 
crests of the latter (in planform) are the dispersal curves of the game. 


For once again the KE are 
£=cos¢, y=sind 
and a simple analysis yields 
cosp=—V,/p, sind=—V,Jp, p=VV2Z4V? (610.9 


8 We have normalized the above sand constant to 1. 
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and the ME, is easily seen to be 
—p+l=0 


so that V satisfies (6.10.1). Also V = 0 on &. 

To construct the solution, note that the RPE contain V, = Vv, = 0, 
which, with (6.10.2) imply that the optimal paths are straight. Denoting 
a smooth portion of @ by x = X(s), y = Y(s), our initial condition 
equation (on @) 

V,=0=X'V,4+ Y'V, 


implies that the paths meet @ at right angles. As distance and time (=V) 
agree on the paths (as speed = 7), we apply the construction of Section 
6.5 to obtain the DS, utilizing simply distance along the interior normals 
to@ as V. As Vis the height of the plot, the latter is a continuous surface 
which must agree with the top of the sandpile. 


Problem 6.10.2. Study the geometry of these dispersal curves. For 
example, with polygonal @, they are comprised of pieces of straight lines 
and parabolas. If a smooth @ has a “vertex” (a point of locally minimal 
radius of curvature) the DS terminates at the center of the osculating 
circle there. 












































CHAPTER 7 


Afferent or Universal Surfaces 


7.1. INTRODUCTION 


As stated in-the last chapter the singular surfaces of the title are of 
type (+, u, +), and the typical one in three dimensions has been depicted 
in Figure 6.1.1b. 

To envisage the role of universal surfaces, abbreviated US, in differential 
games, one can think of such a surface as a union of especially advantageous 
paths. Optimal play will demand that the descriptive point x be brought 
to the US and thereafter remain on it. 

We are dealing with games, in which the essence is that the interests of 
the players are conflicting. Therefore the word “advantageous” above, if 
applicable to one player, can be expected to be the contrary to his opponent. 
Thus, often the decision to utilize a certain surface as a US rests, under 
optimal play, with one player alone. It even may be that his opponent’s 
optimal strategy is perfectly continuous on the universal surface and 
nearby. 

Consequently, much of our investigation will treat of one-player games. 
Little generality is lost thereby, for we can think of the opponent as 
operating under the aegis of his optima! strategy that has already been set. 
Later we shall ascertain this setting and so restore the competitive aspect. 

Accordingly we single out for definition: A ¢-universal surface (or ¢-US) 
is one which is created by a discontinuity in $, while # is continuous on the 
surface. An yp-universal surface (y-US) is defined similarly ($ and y 
interchanged). 

What appears to be the most interesting type of US occurs when the 
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kinematic equations are linear at least in some of the control vari- 
ables (and G also in case the payoff is integral). Let ¢, be such a 
variable. 

Then the idea of limiting ourselves to one-player games can be pushed 
further; we may deal with games with only one control variable. The 
idea is, as before, that we assume all other than ¢, replaced by the ¢,(2), 
$x) which are constituents of optimal strategies. In the resulting one 
control variable game, the main equation will be linear in ¢, as all the KE 
(and G) are. Let A be the coefficient of $, therein. Then generally, under 
optimal play, ¢, will assume one or the other of its extreme values allowed 
by the constraints according as the sign of A. The only possibility of ¢,’s 
assuming an intermediate value occurs at points where A = 0. 

Let us suppose there to be a surface S such that A = 0 on F but no- 
where else in some neighborhood of S. Such loci alone can be universal 
surfaces. However, they need not be. If A does not change sign across 
F, we should not expect F to be singular at all. If it does, Y may be a 
transition or dispersion surface. The former we have seen actually to 
occur in some of our examples; on the latter, on the other hand, the V; 
themselves may be discontinuous and A need not exist at all. 

Unlike most other types of singular surfaces which we treat in this book, 
the US involve no retrograde paths leading to them. Therefore their 
detection cannot follow from the integration of the RPE, and their possible 
locations are largely independent of @ and the initial conditions. 

We do not give a full theory. Indeed the subject of linear vectogram US 
appears big, possibly as big, as our later discussion will indicate, as the 
major chapters of the calculus of variations. We will give an analytic 
necessary condition for the cases up to four dimensions. It suffices for 
many problems, but there appear to be many other interesting facets which 
we leave unexplored. 

Not all universal surfaces require linear vectograms. We take up another 
type first in the section immediately following because of its historical 
contiguity with the calculus of variations. They appear to be of secondary 
interest, and the reader may skip the next section with no loss of future 
comprehension. 


7.2. UNIVERSAL SURFACES WITH 
NULL INTEGRAND 


Supposing, in a game with integral payoff, the integrand G depends on 
the x, only, is 0 on a certain surface S but is otherwise positive. Then S 
constitutes a locus of free rides for the minimizing player P, in that, once 


ł See Examples 7.3.1 and 7.5.1, where A = uV, #0 near the dispersal surface. 
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and the RPE are 


158 UNIVERSAL SURFACES 
on S, travel of x incurs no penalty for him as an increased payoff. Thus, 





clearly, there will exist cases where F will be a ¢-US. > K > 
The analyses of one-player (P) games in such cases is quite straight- t=, V, = 0 

forward. P 
Clearly V will be constant on S. If S meets Y, this constant will be o F, o 

zero. If not, we apply our usual methods, calculating V(x), beginning at y= p ; V, = sgn y 

@ and working away from it until Z is encountered. The lowest value of ao 

V thus found on S will be the constant. From the point (s) U where this where p= J Ve + Ve, 

minimum occurred, the optimal path(s) will leave S. y y 
Then, using as a seat of initial conditions, we find, by the usual and also cos $ = — -£, sin 6 = — 4, 

P P 


integration of the RPE, the paths leading into &. 

Under optimal play, x will traverse one of these paths to Y, then, by 
any admissable route whatever, proceed to U, and thence utilize an already 
found optimal path to @. 

The only possible novelty here lies in our point of view; the substance 
is very old. We illustrate by treating a famous stock problem of the cal- 
culus of variations from the universal surface standpoint. 


The choice of end conditions is at our disposal. The familiar version 
with fixed endpoints we relegate-to an exercise. Our curves will extend 
from some point on the y-axis to a given right endpoint. Thus, in ac- 
cordance with the present theory, we take 

ĉ: xv > 0, o0 < y <o 

€: «=0, y =s. 
Example 7.2.1. Surface of revolution of least area. Given two points in 
the plane both above the x-axis, to find the curve joining them which, 
when rotated about the x-axis, generates the surface of minimal area. 

The solution is classic fare. When the points are sufficiently near one 
another and far from OX, the curve is a catenary. Under sufficient 
negation of these conditions, the curve becomes the union of three 
segments: two extending vertically from the points to OX and the 
segment of OX joining their feet: the well-known “Goldschmidt dis- 
continuous solution.” 


We relook at matters. 
In our one-player game, x moves with simple motion at unit speed. 


First, our standard construction of optimal paths emanating from ¢ 
will yield the classical catenaries. We shall find them in the upper half- 
plane only and so take y > 0, s > 0. 

As on g, V= 0, we have V, = 0 = V, and we complete the initial 
conditions, using the ME, (clearly V, > 0 from the nature of the problem): 


V, = |s| =s, V, =0. 
The integration of the RPE is straightforward, yielding 


V, = S, V, =T 
and 


Thus we take as KE 
+r -—— 
¿= cos $ z= slog +D, y=p paV3+4+7 (7.2.1) 
y = sin ¢. the sought paths. Elimination of 7 leads to the equation of a catenary, 


y = s cosh (z/s). Of course, the original phrasing of the problem requires 
that s be selected so that the path (7.2.1) passes through the prescribed 
starting point. 

We will need below 


Since the elementary formula for area of revolution generated by an 
arbitrary curve is 27{|y| ds, we take 
G = ly! 
r 1 + F 
v={ pdt = I T+ s$ lo ete) 
; 21? 8 Is] (7.2.2) 


Now for our major interest, the d-US. The curve Z on which G = Ois 


so that our payoff will be proportional to the desired area. We use |y], 
not y, so that we can use as & the entire x, y-plane. 
The ME, is 


—p+ly| =0 t=", y=0 









































160 UNIVERSAL SURFACES [7.2] 


3 








Figure 7.2.1 


where u, a parameter, > 0. The ME, shows that on S 

Va = V, =0. 
(As F meets Y, on the former, V = 0.) Integrating the RPE with these 
initial conditions gives us, for paths emanating from /: 

V,=0, V,=07r, p=T 
e= pM, Y= or. 

Here o = sgn y and clearly we use both o = 1 and —1. These “‘tributaries” 
to F are clearly vertical lines. We compute directly that on them 

V = 47° = by’. (7.2.3) 

The two sets of paths join on a dispersal surface which is obtained by 

equating the two values of V, (7.2.2) and (7.2.3); in the upper half-plane 


etr], 


S 





ty’ = alor + s* log 
If from this equation and the pair (7.2.1) we eliminate s and 7, we reach 
the equation of the upper dispersal surface, which turns out to be 
x= cy. 
Here c is a constant satisfying 
cexp #1 + c?) = 1, (e = .53...). 
The optimal paths are sketched in Figure 7.2.1. 
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Exercise 7.2.1. The classical problem with two fixed endpoints. Let 
Q = (a, b) and we use for € 


z =a + ôcoss 


l y =b + dsins 
so that 
ô(— V, sins + V, cos s) = 0. 


Proceeding in our customary manner and taking the limit as 6 — 0, we 
have on @ 
V, = b cos s 


V, = b sins. 
Complete the solution using these initial conditions, 


Research Problem 7.2.1. The two-handed version. Let Q above move 
with simple motion, but with speed <1, under the control of E, so that 
the problem becomes a pursuit game. Under certain circumstances, 
clearly E can control which type of path (catenary or 3-piece polygonal 
line) P will follow. Does the optimal solution reflect this control’s being 
a major factor in E’s strategy? 


7.3. UNIVERSAL SURFACES WITH LINEAR 
VECTOGRAMS: AN INTUITIVE PURVIEW 


To illumine our general ideas we consider one-player, two-dimensional 
games. Such: must necessarily have integral payoffs if there is to be a 
universal surface. For, if the payoff were terminal, we know that the 
Value is constant on each optimal path. Then it is constant on the US 
and must retain the same value on all tributary paths; thus it is constant 
in a region containing the US. Accordingly in this region all strategies 
are optimal, and the US exists only in the most trivial sense. 

Having settled on an integral payoff, we shall assume, in virtue of the 
last section, that G # 0. Let us take for definiteness G always positive. 
It will simplify matters if G is independent of the single control variable, 
$, but we shall soon see that the following ideas hold when G is linear in $. 

Suppose in a region of &, the gradient of V exists and is not zero. 
(Note that this is not true near a US of the null G type.) Then we will show, 
in a rough geometrical way, that a US can arise only if the vectograms are 
linear. 

Let Figure 7.3.1a depict a typical linear vectogram. Imagine it drawn 
to a very small scale so that its vectors closely approximate the actual 
possible displacements of x during a very short interim. Superposed on the 
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(a) (b) 


~~ <7 
(c) 
Pie © a 
(d) 


Figure 7.3.1 


same figure are some of the curves on which V is constant; V is supposed 
to increase in direction of the arrow labeled V. 

Which vector of this vectogram is best? Obviously the one achieving 
the greatest decrease in V or the one penetrating farthest through the pencil 
of curves. In this case it is the (over-scored) leftmost vector. 

In (b) we have a similar situation, but now the greatest decrease of V 
accrues to the rightmost vector. 

It is clear that the criterion is the direction in which V increases along 
the headline (the line of the arrowheads). Thus an intermediate vector 
can attain maximal penetration only when this increase is zero or when 
the headline is tangent to the local curve of constant V. Such a case is 
depicted at (c). 

Suppose there is a curve on which such occurs at each point. For 
example, the dashed curves of Figure 7.3.2, where it is supposed that all 
vectograms have horizontal headlines. Such curves alone are eligible to be 
universal. Let us now return to Figure 7.3.1c and suppose that the point 
depicted lies on such a curve. From our study of (a) and (b) of this figure, 
we see how the vectograms immediately to the right and left will behave: 
the small vectograms drawn at (c) have their optimal vectors overscored. 
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Thus the optimal paths from both sides will converge to the center and we 
will have a US? 

To justify our assertion that linear vectograms are essential, observe a 
typical properly convex vectogram such 


as appears in Figure 7.3.1d. Again the ray 
criterion of matching the slopes of a 


headline and V-curve holds; in the 
figure the overscored vector yields max- eS 
imal penetration. Thus, generally, the 
minimizing -¢ is an interior one and va . vie 
varies continuously with x so that no l 
US can appear. If the vectogram is but 
slightly convex, we may expect that the 
optimal paths will resemble those of the 
linear case, probably in some manner 
such as that suggested by Figure 7.3.3. 
George Dantzig has suggested ex- 
ploiting this idea to compute a US. 
Render a linear vectogram slightly con- 
vex, ascertain the paths, and then study 


oe 
wore 
Sey 


sA 


their limiting behavior as we allow the Figure 7.3.2 


convexity to disappear. 

Let us seek some heuristic insight as to what may happen when more 
control variables are present or at least considered active. Figure 7.3.4 
depicts a three-dimensional analogue of the subject of Figure 7.3.1. The 


A ODS 


Figure 7.3.3 


* Should the V-curves curl upwards, however, the little vectograms would have to be 
interchanged and we would have a dispersal surface. But generally at such the V; are 
discontinuous and the V curves suffer a saltus in their slopes. We invite the reader to 
attempt the construction of an example in which there are smooth )-curves at a 
dispersal surface. 
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Figure 7.3.4 


vectograms are cones and linear in the sense that their bases are flat (the 
headplanes). Also drawn are some surfaces of constant V. The same 
argument as before shows that an afferent behavior may possibly occur on 
the curve on which the headplane of the local vectogram is tangent to the 
V-surface. Then we have a “universal curve” rather than a surface. 

It seems plausible that this is an instance of a general truth: vectograms 
linear in several control variables can lead to universal manifolds of lower 
dimension. It is interesting to think of the possibility of such a manifold’s 
consisting of the intersection of a number of US each relating to a distinct 
control variables. We have not explored the ample terrain which appears 


to spread before us here. 
An example may elucidate our feel for the subject a bit further. 


Example 7.3.1. Let us consider motion in the upper half-plane with P 
endeavoring to have x reach the z-axis in minimal time. All the vecto- 
grams are to be shaped as in Figure 7.3.5a, but their size is a given smooth 
function u of x and y. That is, the KE will be 


å = gu(x, y) 
j = —u(x, y), -l<¢<l, u>0. 
Let us think of the surface which is the graph of u as being roughly 


indicated in the figure, where the curves are meant to be sections perpen- 
dicular to the page. That is, U, and U, are crests of ridges while D lies at 


the bottom of a valley. 
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Clearly the optimal paths, when nonsingular, will have slopes of +45° 
As U; and U; are loci of large u, we may expect them to act as high speed 
arteries and be universal. Similarly, D, a curve of low Speed, might be 
suspected of repelling x and being a dispersal surface, 

The first conjecture is correct as will appear later (Example 7.5.1). That 
the second is generally false* can be gleaned from Problem 6.10.1 (with 
a= 0). Of course, if U, and U, are both US, a dispersal surface must lie 
ae between them but not necessarily at the bottom of the 
valley. 

Observe how x, if starting from such a point as X;, might first go to U, 
traverse it downward to its end, then follow a —45° path to U, and 
traverse it to X,. 

Because U, and D terminate, there must be changes of sign of u, on the 
left part of the plane. These lead us to suspect a transition surface there. 
on the grounds that P might gain by successive diversions of x toward 
regions of higher u. 

A later result will show that the universal surfaces are marked by the 
condition 


U, = 0 (7.3.1) 


but no such general criterion can distinguish dispersal or transition 
surfaces, for they depend on the & and the initial conditions. 


AN 
Xı 


U2 


X2 
Figure 7.3.5 


a It is obviously true sometimes, for example, if D is vertical and an axis of symmetry 
of u. 
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7.4. THE ANALYTIC NECESSARY CONDITION FOR A 
LINEAR VECTOGRAM TYPE UNIVERSAL SURFACE 


We deal with a one-control variable (and hence a one-player) game. It 
may and will be assumed to have terminal payoff, as shown in Theorem 
2.4.1. At the outset of the preceding section we learned that we must have 
n (the dimension number) > 3, if a US is to occur. 

The KE will be of the form 

= u$ +a (isl,...n (7.4.1) 
with «, and £; given functions, assumed smooth as need be, of the z;. The 
vectors « and # are supposed lineally independent. We may always take 


for the constraints -1<¢<1 (7.4.2) 


by choosing for £ the vector extending to the center of the headline and « 

the vector reaching from there to one of the ends of the headline.* 
Suppose we ‘have some definite game, with Value function V (x), which 

contains a US on which V is supposed smooth and the ME holds. We will 


have $= o = —sgnA (7.4.3) 
where 
A = aV, 
2 ; 


whenever A # 0. Similarly, we denote 2,8,V, by B. As the universal 
surface must be traversed by an intermediate ¢, say $ with —1 < $ < 1, 
on it we must have A=0. (1.4.4) 


Since the ME is A$ + B = 0, on the US we must also have 
B=0. (7.4.5) 


Now the tributary paths to the US are obtained by integrating the RPE, 
utilizing the US and the values of V on it as initial conditions. As there 
are to be distinct sets of these paths from the two sides of the US, it can 
only be that 6 = +1 for one side and —1 for the other. From (7.4.3) 
then, A is of opposite sign on the two sides. 

The RPE of the tributary paths are 

č; = —a,0 — Ê; 
Ŷ, = > Vays + Bu) 
a 
where «,; means 0a,/02, etc. 


4 For certain problems, where other constraints are more natural and convenient, 
our procedure can easily be modified. 
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Let us see how A changes as we move away from the US on either side. 
A= Dd Ves F V4) 
3 
= 2 [> Vileo + Pidas + > Via;(—o,0 ~ A) 


the latter step utilizing x; = 2,0,,2,. If we reverse indices in the latter sum 
of the bracket, we have 


i= =2 (tibi — Pita) Vi (7.4.6) 


But this expression is independent of ø and therefore A is same on both 
sides of the US. Then it must be zero, for suppose, say, A>0. Asd=0 
on the US, A would then be positive on both sides at a sufficiently short 
distance away. 

Defining (these quantities are basic) 


Yi = 2 («bs — Pija) (7.4.7) 


the essential part of the sought condition is thus } y,V, = 0. We have 
proved i 


THEOREM 7.4.1. For a game with one control variable ¢, terminal 
payoff, and KE (7.4.1) on a universal surface on and near which V is 
smooth® we must have 


A= Ya, =0 
B= 2 BM =0 (7.4.8) 
C= Drv, =0 


From the point of view of semipermeable surfaces, we know that the V; 
can be considered as the components of the normal vector v. Thus a purely 
geometric version of this result runs: 

If a region of @, filled by a family of semipermeable surfaces and 
traversed according to the optimal ¢ (which forestalls penetration), 
contains a US, at the US the normal » to any such surface must satisfy 


2, an = È fo = È ym = 0. (7.4.9) 


THEOREM 7.4.2. The V, are continuous across a linear vectogram type 
universal surface. 

Proof. We choose coordinates so that the US lies in the plane: x, = 0. 
We suppose a proper US, that is, on it the vectograms do not lie in its 


5 Theorem 7.4.2. renders this hypothesis unnecessary. 









































168 UNIVERSAL SURFACES [7.4] 


plane; in the present coordinates this means that «,, 8, cannot both be 
Zero. 

The US consists of a family of optimal paths and so V will be known on it, 
By differention so will V}, . . . , V,. Finally, V, is determined hereon by at 
least one of the equations, A = 0, B = 0, for at least in one the coefficient 
of V, does not vanish. 

Now, taking ¢ in turn as +1, we can integrate the RPE using initia] 
conditions on the US of which the above values of V; are part. The result 
will include the values of V and the V; as functions over some neighbor. 
hood of the US. From the known theory of first-order partial differential 
equations, and their integration by characteristics, the V; obtained will be 
the appropriate partials of V. It is also clear that this V is the Value. 

Then, as asserted, the V, are continuous across the US because at it, 
from both sides, the V, approach their unique initial values on the US. 


Research Problem 7.4.1. Investigate the continuity of the second partials 
across a ¢-US. 

There are two benefits of the continuity of V, at a ¢-US. One is that the 
ME is everywhere satisfied. Thus, techniques of proof, such as the veri- 
fication theorem, apply, and we have the same assurance of the validity of 
a solution as in a case free of singular surfaces. 

The second concerns the control variables other than ¢. We have stated 
that, when concerned with a $-US, we can suppose them as already fixed at 
their optimizing values as dictated by the ME,. Such values will generally 
be rather simple functions of the x, and V,. The continuity of the latter 
gives us some assurance that the other control variables will be continuous 
at the ¢-US. 

These conditions are necessary but, as we shall see, by no means 
sufficient. Therefore we shall define a candidate for universal surfaces 
(CUS) as a smooth surface which 

C1. satisfies (7.4.9) or (7.4.8). 

C2. is the union of a set of paths satisfying 


ù; = ad + B; for some $, with —1 < ¢ <i. 


7.5. THE WORKABLE CONDITION WHEN n=3 


The conditions (7.4.8) (or (7.4.9)) do not directly instruct us as to how 
to detect CUS, for they involve the V, (or »;) which are not known at the 


outset of a problem. 


€ See for example, Reference [6], Ch. 2, §3.2 and 7. 


Particular surface, this surface ma: i i 
art , y well be a third variet 
trivial nature; (7.5.1) will hold for it clearly. á 


[7.5] THE WORKABLE CONDITION WHEN 
n s 


When 7n = 3, however, matters are i 


mmedi ; A 
components of v are not all 0, ona CUS we ediately simplified, <. 


must have at once the condi 108: 
a By yF | 


Xa bz Yaj =0 


a Ps ys 


We pie here an equation in the x; which may be the e 
several surfaces. Each such on which C2 also i 

Our procedure will be, whatever n, as Ai T 
solution suppose we encounter a region of & around which the Be a 
paths disperse and leave a void, that is, a region without path sean 
the shaded area of Figure 7.5.1a (when n = 2), We seek a dus eae 2 
we find one AB as in (b) within the void. The tributary paths will blend 
smoothly with those previously found. This is because the linearity of fa 
KE in ¢ imply that those RPE which equate the 2, are free ahi v 
Thus for a definite value choice of o, a path is determined only fro : the 
initial values of the x, and so paths filling a continuum such as CAB in ) 
of the figure will constitute a smooth family. oe 

le the void is filled by paths and hence also bears functions ¢, V, and 
e hat assurance have we that they constitute the correct solution? 

e have already proved the continuity of V and its partials acr 

the US. This continuity also holds at the junctions such as AF and 4G in 
the figure, for the V, can be obtained from integrals of the RPE and the 
initial conditions, even though partly on @ and partly on the US = 


(7.5.1) 


quation of one or 





(b) 


Figure 7.5.1 


z : : 
We have stipulated the linear independence of « and B. Should such fail on a 
of US, but of a comparatively 
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continuous. Thus we have a solution of the ME throughout and techniques, 
such as the verification theorem, are valid. 


Example 7.5.1. A previous illustration. We apply our criterion to the 
problem depicted in Figure 7.3.5. Changing the notation slightly the 
KE are 

$, = pulty, to) 

ty = —U(2;, tə) 

dy = | -l<¢<l,uz0. 


The third equation arises from our usual transformation of an integral toa 
terminal payoff (see Section 2.4; here, of course, G = 1). Thus we have 


a = u, ag = A = 0 
and f,=0, ba =—u, B= 1. 
Recalling y: = X (sBs — Busts) (7.4.7) 
we compute Yı = %ı2fa (all other terms being 0) 

= —uu (un = 2) 
Ox, 
Ya = — fan% = uu 
¥3 = 0 


so that the determinant (7.5.1) is 
u 0 —uu 
0 —u  uų|= uu 
0 1 0 


and as u > 0, our condition is (see (7.3.1)) 


It clearly holds on U}, Ua, and D. 

Of course, no more is claimed here than the usual type of “necessary 
conditions” common in minimizing problems: derivative equals zero at 
minimum of a function; Euler equation at minimum of an integral. 


Example 7.5.2. Shortest route of a car, boat, or airplane to a fixed destina- 
tion. The craft in question moves with a fixed speed, but its curvature is 
bounded by a given 1/R. It is steered by choosing at each instant an 
admissible value of this curvature (corresponding to a steering wheel or 
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rudder with instantaneous response). We wish to pilot the craft from a 
given starting position and attitude to within a given distance / of a given 
destination point A in as short a time as possible. Or, because of the 
constant speed, we can equivalently speak of the path of least length. 

It is clear that we have a special case of the homicidal chauffeur game 
with an immobile evader. The KE are then the same but with We (E’s 
speed) equal to zero. We replace w, by w and also write æ, x, in place of 
z, Y. 

The KE, given in Example 2.2.2, then become 


a=- 


and, to attain a terminal payoff in the standard way, we adjoin 
ds = G = l. 


In this 3-space, @ is the cylinder of radius / and axis Oxs. The standard 
procedure applied here will yield ¢ = +1 for the right (æ, > 0) and left 
sides of the upper surface. On any 2,-section of @ the paths will be arcs of 
concentric circles centered at (+R, O) as shown in Figure 7.5.2a. Such arcs 
correspond to sharpest possible right and left rudder, and the paths leave a 





(b) 


Figure 7.5.2 
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Figure 7.5.3 


void (shaded in the figure). Let us apply the CUS criterion. We have 


Ww 
a= - (0 bi =0 


A = (£) Bb, = —w 


a = 0, bs =1 


and so, omitting terms which are zero, 


Vi = MBs = — (e) Y2 =y=0 
2 
= (2) a0 = 
R R 


w? 
oa = of- Ee 


0 i 0 


which leads to 


The plane x, = 0, being navigable by $ = 0, isa CUS. Returning to the 
two-dimensional version (x, %-plane with G = 1), we use as initial 


conditions : 
z,=0, z, =1+s(s>0), hes re 
These are employed with RPE in our usual way, and they lead to tributary 
paths which are also circular arcs concentric about (ÆR, 0). 

The formal details of integrating the RPE, working out y, etc. we leave 
to the interested reader. Instead we note the simple plausibility of what we 
have found. l 

The US, as $ = 0, corresponds to straight travel. The tributary paths 
are sharp turns which precede such a state. In the realistic space, this 
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means that the craft turns sharply until pointed at the target 4 and then 
proceeds there on a straight course,® as shown in Figure 7.5.3. 


7.6. WHY THE NAME UNIVERSAL SURFACE? 


Through problems of the ilk of the preceding. One is startled at first, 
when working with the ME relevant to, say, steering a boat, to learn that 
the formal mathematics would seem to dictate always sharp right or left 
rudder. One knows better; boats are not piloted this way. On a long 
(ideal) voyage on a uniform sea, on the other hand, the rudder should 
almost always be straight. 

In the last example, the US is a set of measure zero in the reduced space 
&. Yet it alone marks straight travel and, despite its nullity of area, it is 
the bearer of all long voyages save brief initial maneuvers. 

The possibly pretentious name “universal surface” was intended to 
direct attention to the importance of these loci. 


7.7. THE CALCULUS OF VARIATIONS VIEWPOINT 


The universality of US is further clarified as their connection to the 
Euler equation becomes apparent. We adhere to the three-dimensional 
case here and will resume this aspect of the subject in a later section after 
the required additional analysis. 

Let us suppose we have a game with integral payoff in the plane with G 
linear in ¢. That is, the KE are 


= = ud + fı 
Y = te = ah + By 
and G = ash + Bs. 


(x; 8; = functions of x; y only) (7.7.1) 


As was done earlier (Section 5.3), we can cast all in the calculus of 
variations mold. Writing 
dy = ah + Be 
dx ad+ By 


we solve for ¢ and substitute the result in 


gh + b m 
| ane md (7.7.2) 


* We have not solved this problem completely, of course. There are interesting 
Possibilities when A is initially within one of the minimal turn circles. See the homicidal 
chauffeur problem, of which, as remarked, this is a special case. 


y = 
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Figure 7.7.1 


We have reverted to the classic minimization of an integral of the type 


fr (x, y, y’) dz. (1.7.3) 


The Euler equation of this integral turns out to be of zero order. That 
is, it is not a differential equation at all but agrees with the US condition 
(7.5.1). 

Exercise 7.7.1. Carry out the preceding operations in detail and obtain the 
F of (7.7.3). Show that the classical Euler equation 


d 


a Fy, — F, =0 
here is (st e) + (a =0 (7.7.4) 


and that this equation is equivalent to (7.5.1) when the «,, B, are free 
of x5.° 

The role of our analytic condition for US is now clearer. It is subject 
to the same intracies of “necessary and sufficient condition” type as fill so 
many pages of most calculus of variations texts. In Section 7.12 this idea 
will be discussed further. 

As an instance of how (7.5.1) may facilitate calculus of variations 
problems, we consider 


Example 7.7.1. Parking a car. The car moves asin Example 7.5.2. It is 
located in the large unobstructed parking lot of Figure 7.7.1 and is to park 


° Recall that we postulated linear independence of « and $ and therefore the de- 
nominators in (7.7.4) + 0. 
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at the line L, its terminal inclination of L being the given angle u. It is to 
do so in the least time. 

We can confidently expect that, if the initial distance from L is at all 
great, that most of the optimal path will be straight. The novelty here over 
Example 7.5.2 is that final as well as preliminary turning maneuvers are 
required, the former to attain the angle u. We confine our interest to the 
following question: Should the straight part of the optimal path be per- 
pendicular to L or, in anticipation of the final turn, should it slope a little 
p-wise (such as the dashed path of the figure)? 

We use the reduced coordinates x,, x, as shown in the figure. The KE 
are then 


%, = —W COS x, 

a= (i 
and, as G = 1, we adjoin 

t = 1. 


Calculating the y; we obtain the table 





i Xi B; Yi 
1 0 —wcos £} (w°/R) sin 2, 
2 —(w/R) 0 

0 1 0 


The condition (7.5.1) then is 


3 
_ (5) sin £% = 0. 


Any horizontal line is then a CUS. Our question appears to be answered 
in favor of the former alternative: aside from the initial and final turns, 
the car approaches L perpendicularly. 


7.8. ALL STRATEGIES OPTIMAL 


There may be regions in & in which any value of ¢ serves to achieve the 
Value of the game. We will limit ourselves to games of the type describable 
by (7.7.1). 

The main result will show that a condition is the identical satisfaction of 
our CUS condition. 

The ME is 


nun le Vn + aV, + aslo + [PVs + BaV, + Bs] = 0. (7.8.1) 
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If in a region Z c &, all ¢ are optimal, then both brackets equal zerg 
there. We have simultaneous equations for V, and V,, and their solution į 
y. — tbs — Osho y y. — %61 — aps _ —N. 
í aiba — aapi 5 aipa — aapi 
Hence we must have identically (V,, = V,,) 
M,+N,=0 
but this is the form (7.7.4) of the US condition. i 
Conversely if this condition holds in a region &, then there must exist 
there a function V(x, y) satisfying (7.8.2) and hence annulling both the 
brackets of (7.8.1) and hence yielding identical satisfaction of the ME. Buj 
this function will be the Value of the game only if its partials jibe with™ 
those of the known Value on the boundary of 2. 


Example 7.8.1. The vectograms are all of type shown in Figure 7.8.1; 
G = 1 or payoff = time to termination. That is, the KE are 
i a= ¢, -l<¢<l. 
If ¢ is as shown, then it is immediate that 
in © any ¢ is optimal 
in@ ¢=1 
in@ ¢=-1 
in @ termination is impossible. : 
Yet the condition (7.8.3) holds everywhere, for all «;, 8; are constant, ‘ 


y= —1, 


Figure 7.8.1 
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search Problem 7.8.1. Investigate the relation between the lack of 
ptimal paths and the more general CUS conditions of this chapter. 


. THE WORKABLE CRITERION WHEN n=4 


When n; the dimension of & or number of state variables, exceeds 3, the 


= pasic conditions 


Lay, = Dipoi = Lyy, = 0 (7.4.9) 


where » is the vector normal to surfaces of constant V at the CUS, no 
longer determines a CUS directly. In this section we shall exhibit, when 
n= 4, a technique, entailing differential equations, for locating such 
surfaces. This is as far as we shall go with generality, but subsequent 
examples will display a few sporadic instances where we construct US with 


n> 4. 
The formal result will of the type 


O¢+R=0 


where Q and R are functions of z4, . . . , £4. Thus if Q = 0, the equation 
R = 0 will be that of a possible US. But if Q # 0, (7.9.1) will determine 
¢ as ¢{x), a function which when inserted into the KE 


t, = ad + Bi i= 1,2, 3, 410 


will render them differential equations. Since the totality of their integrals 
can be expected to fill more than a surface, appropriate initial conditions 
must be selected. It is likely that they will be supplied naturally by the 
problem at hand. 

Let a game with one control variable, ¢, and terminal payoff have a US. 
An artery will be defined as the intersection of the US and a surface of 
constant V. Thus an artery is a “curve,” that is, it has dimension n — 2. 
The surface of constant V containing it consists of the artery and the two 
sets of tributary paths (those with 6 = +1 and —1) feeding into it. 

Let a surface F of constant V be given by 


(7.9.1) 


(7.4.1) 


z; = Xs, t, 7) (7.9.2) 


where s, t, r are parameters enjoying the following special properties: 
The retrograde time parameter, as usual, on the tributary paths is r. 
The artery of S is given by 


z; = X:s, t, 0). (7.9.3) 


1 In this section such indices will hereafter be understood to run from 1 to 4 without 
specific mention. 
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Observe that (7.9.9) holds also when w = 1, for then it is 
2 A,X AX j =0 


Navigation in the US is accomplished with ¢ = $ and the time parameter is 
t. That is, the artery consists of the paths (7.9.2) where s is held constant 
and r held 0; thus when 7 = 0 


which is true, for A,, is a skkewsymmetric matrix. 
Now, if F is to be a nondegenerate surface, one of the three minors of 
the “Jacobian matrix,” 
(2), w=s,t,7 
ow t ia 


is not zero. Let it be the one obtained by deleting the column where 
i= k. Then, multiplying (7.9.6) by 


Z An(asd + i) 
and (7.9.9) by », and subtracting, we obtain 
z [30 Ars — % Ayah + B] Xw = 0 (7.9.10) 


x,(= 2x) =a, + bi (7.9.4) 


The normal » to S must satisfy (7.4.9). This means that, up to an 
arbitrary multiplier which is a function of the x; the v; are the 3-minors 
(with the appropriate alternating signs) of the 4 x 3 matrix 


(a Byi)- (7.9.5) 


By choosing such a determination, we suppose the v; are known functions 
of the z,. 
As the v, are normal to S 





Li: Xiy = 0 (7.9.6) 
where 


Ox; , oe ; 
Xi =— and w=s,t, Orr. where the prime on the sum signifies the deletion of the index value k. 
ow Because the determinant | X; „lixx = 0, we must have satisfied the condition 


we will call 


If w = s or 7, differentiating (7.9.4) with respect to w gives 
Ca: 2% Ars ~ Pr Ag\(ajb + B) = 0. (7.9.11) 


Xitw = Lass + Bi) Xiw + thu 
i This is the condition (7.9.1) promised earlier. 

Actually all the conditions Cy, (for i # k) turn out to be equivalent. 
This follows from the purely algebraic lemma below. To apply it we must 
first show that 


We also differentiate (7.9.6) with respect to t and use the above for Xis: 


È Xul vag + By) + Z woh + Bi) X jw = 0. (7.9.7) 
7 r] i, 


3 A = 0. 9. 
Note that ¢,, does not appear as 2 tp = 0 (7.9.12) 
This follows from 
Zam = 0. 
' 2 ny, =0= 2 napi — Bisa) 
t 
lity also implies, on differentiating with respect to z; 
T EE a a de p p and we make replacements of type (7.9.8), which remove the «,; and bi, 
Dd va = —> 0%; (7.9.8) and then reverse indices is one of the sums. 
3 i 


LEMMA 7.9.1. Let a, 8, » be vectors in 4-space such that 


and, similarly, when £ replaces x We make these substitutions in (7.9.7) 
and reverse indices in the second sum, leading to 2 ay, = 2 By, =0 (7.9.13) 
> ka [2% — vy )(a;¢ + B | = 0. (7.9.9) and « and £ are lineally independent. Let A = (A,,) be a skewsymmetric 
i j 4-matrix such that 


Define Ay = Vij em, Vig > Aub; = 0. (7.9.14) 
ii ‘ 
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Define 
a Si A A,,) (7.9.15 
= Vu Ang — Yy Aus). 9. 
Bus 7 B; 4 v 4 ) 


Then for any two pairs of indices (u, v), the pairs Aus» Bus are lineally 
dependent. That is, either all A,, = 0 or else some do not and the ratio 
B,,,| Auy is the same for all such, while B,, = 0 for the rest. 

Proof. Let us perform a rotation with matrix R = (r,,) in the 4-space 
containing «, f, and v. Thus if'é, B, ? represent these vectors in the new 
coordinates, we will have « == Rā, B = Rf, v = Rö where the first relation, 
say, Means 

a = x Tij% 


The matrix itself satisfies RR’ = E or rufus = riuju = Ôu The 


u u 
equivalent of A in the new coordinates is A, where A = R AR’ or, when 


written out, 
Ai; = 2 Å isje 
8, 


One can verify standardly and easily that 
2 A, aĝ, = 2 Ab; 


and that A is skewsymmetric. 
We next show that the skewsymmetric matrices A and B transform in the 


normal way, that is, as A does. 
Substituting into (7.9.15) 


Aw = > (pp) u? Etor i) el 
5.9.0,8,t 
where the dots stand for the same expression with u and v interchanged. 
Summing over j gives 
Aw = È Pe A,r ualvs — Toa! us) 
Q8, 
= 2al A aes Ps Aa) uav 
@,8, 


=> Å a uto (7.9.16) 
@,8 


and a similar relation holds for B, f’s replacing « being the sole change in 
the proof. 

If v = 0, the lemma is true trivially as all A,, and B,, = 0. If v 49, 
we can now select the rotation R. Choose the first axis along ». The 
second is chosen along «, which is allowed by (7.9.13). Similarly, we can 
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choose the third in the plane of « and f. Then in these new coordinates 
ř = (7, 0, 0, 0) 

&% = (0, &, 0, 0) 

g B = (0, Bs, Bs, 0) 
with ři, Zes 8; = 0, the latter two following from the independence of«and 8. 
Now (7.9.14) is 
i Arstishs =0 
implying 
Ags = Ase = 0. 
Note that 4,,, = O when u # 1 and v # 1 and, from the skewsymmetry, 
Ay, = 0. In the remaining cases we have (let us take ñ, = 1) 


Ay = —A,, = & Å, 


so that 
Ay, = 0 
Ay = 0 
Ay = ð Au. 
Similarly, 
By, = -Dyu = A Aye + Bs Avs 
and so 
By, =0 
B,, = 0 


By = Ba Aue + Bs Aus. 
Thus from (7.9.17) 
Aw = Aul usva = usla); 
similarly, 
B,,, = By, x (the same factor) 
and the lemma is proved. 
To summarize our conclusions let us put 
Ui» = vy A,; — VY Aus 
where as before 
Ais = Vi — Vje 
We have established 


_ THEOREM 7.9.1. In a game with one control variable, ¢, terminal payoff, 
linear vectograms, and n = 4, a CUS must be a surface on which the 
equivalent conditions C,,, (u # v) hold and not all the v», 4 0, where Cw 


means ` 
(= aUih + (2 B,Ute) = 0. (7.9.17) 
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A first example is in the form of 


Exercise 7.9.1. For the KE 


a= (Sttogn) y+ 


Ta 
Èa = Xiah 
dy = t$ 
dy = hy 


carry through the computations and ascertain that the equations Cu, 
consist of = 
(x5 log 7) — 2° 


equated to zero after being multiplied by the factors in each case: 
Cy: — 2, Caz? — Zt log 2% 
Cis: Tts Ca: — 95 a4 
Cy: 0 Coa: Tata. 
To obviate the ambiguity in the determination of the », take », = 2%. 


Problem 7.9.1. Note that in the previous exercise, x, appears only in the 
fourth KE. Such is tendentious of the first three equations forming an 
independent system. Treat this trio by the method of Section 7.5 and 
obtain the CUS. Note that such satisfy the differential equations of 
Exercise 7.9.1, that is, the KE with $ replacing ¢, but do not seem part of 
the formal general solution. Are they singular solutions ? 


The v; in our work are of arbitrary length. If they are all multiplied by 
an arbitrary function, the intermediate steps will be altered; for example, 
there will be terms added to the A,,. But that the final result is un- 
affected we leave the reader to show in 


Problem 7.9.2. If the v; are replaced by hv, where h is a differentiable 
function of z,,...%,, show that the C„, each become multiplied by h’*. 


Research Problem 7.9.1. Our technique entails essentially the six equations 
Cus Which we proved equivalent. One suspects that there should be a 
more direct approach eschewing this polychotomy. Is there? 


Example 7.9.1. The classical brachistochrone with bounded curvature. Let 
us once again return to our vehicle with bounded curvature but let the 
speed now be a given function of x and y. Then the reduction of co- 
ordinates in Examples 7.5.2 and 7.7.1 is no longer possible. 

Let us take x, and z, (x and y) as coordinates in the plane. If we let the 
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speed be J ta (see Example 5.2) we have, aside from the curvature con- 
straint, the classical calculus of variations brachistochrone problem. If 
æ; is the inclination of the velocity vector of the z-axis, the KE are 

= V Lo COS 23 

ty = y Ta sin T3 

ža = —(V tR, —-1<¢<1 

=l. 

Tabulating «;, i and y; after computing the latter from (7 4.7): 
Yi = È (aub; — pit) 
Ci B; Yi 


i 
1 0 Wi Ta cosz,  (£a/R) sin x, 
2 0 Vaa sin va ——(%/R) cos x 
3 
4 


—Vx/R 0 ($R) sin wy 
0 1 0 
For v we use 
Vy = —COS%s, Vg = —SiN Zg, %=0, n= Vita 


which is seen to be orthogonal to «, $, y. After ascertaining the A,, and 
then the U;,* we find for Cy, 





[As a check, C,, turns out to be 


z) Toy 1 
—{—]dcos 2, — -= sin" z}, + - = 0, 
(3 aa 
the same thing.] 
Thus ¢ = R cos 2/27, and we are led to the differential equations for 
the artery: 
t= Va, COS 23 
ty = V Ta SiN 23 
Žž = — COS za2V t, 


ty = 1. 
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The general solution of the first three is, as may readily be checked, 
zı = cı + K(@ — sin 0) 
£a = K(1 — cos 6) 
£ = (7 — 0) 


where 6 = (t + c,)//2K and c;, cs, K are constants. 

Note that the first two are the equations of the classical cycloids. We 
have here an instance of a general principle which will be elucidated in 
Section 7.12. 


Exercise 7.9.2. A simplification of the last example. The point x moves in 
the plane with constant speed w but with curvature bounded by 1/R. The 
payoff is the time to reach a given curve @. The arbitrariness of @ pre- 
cludes reducing the coordinates to two, so that this problem must be 
treated in the manner above. 

1. Show that CUS are straight lines. 

2. Show, by taking into account the initial conditions, that a further 
condition is that these lines be perpendicular to @. The ideas of Section 
7.10 will be helpful. (Similarly, one can attain this same perpendicularity 
condition for the above cycloids.) 


Example 7.9.2. The Battle of Bunker Hill. This problem belongs to a 
later chapter. Here we confine our attention to the computation of a US. 
We need but know the KE, which are 


a = — CoLaPo(%4) 


ty = — CX p(t) 
vs = —c$ 
č = —1 (7.9.18) 


with 0< ¢ <1; pı and p, are given functions of %4; Cı and c, are 
constants. 

Writing the «, and f, in the following table and then completing it by 
computing the y; we obtain 











i X; b: vi 

1 0 — Ctp — CC pp 

2 —cy2py 0 Calcat pipz + tpr] 
3 —C 1 0 0 

4 0 —1 0. 
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The simplest v, appears to be 


Vy = CP pa + Gipi 

Va = Col PiPo 

Vg = — CT Pp: po 

Va = — Cpa Cappo + Xp). 
Cıa Say, leads to an apparently formidable result: 
¢ = ees [2c ta pi Pa + cyt, ta PPa Pi 


+ x Da +2 12 n], 
But if we put 1 (PaPi Pa + 2PoPs — papapi) 





Z (7.9.19) 
where 


The differential equations for the artery in retrograde form, become 


o 

Zi = CXP 
Ta = %p,Z 
O 

Tg = 

o 

y= 


We are going to integrate with the initial conditions 
a, = si = 1, 2, 3), 5 = 0. 


Thus we will have x, = 7 and to the ° and ’ derivatives are the same. 
Now 


vy zı 
a [z 7 air) _ up,’ _ pW 
Pa Pi 2c 
and this equation integrates easily to 
a) 
u = p| K ——}. 
r,( 2e, (7.9.20) 
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Here K is the integration constant and is easily seen to be 
K= (e) pre (7.9.21) 
s,/ p,(0) 2c 
Putting Q(7) = Í P:(®)p:(x) dx, we easily perform the full integration: 
0 


a, = Oh eKO 
Pı 
Z_ = ux, (use the above values of u and z) (7.9.22) 


7 12 = 
sakia Í Pi dr — WOWO Ls 
4c, Jo Py Pe 2ca 
Ta = T. 


Example 7.9.3. The war of attrition; second version. Again this is a game 
whose significance will be discussed later; here only the formal analysis of 
its US will interest us. It entails a short cut. 

This time it turns out that the KE lead us to the table: 








1 a, B; Yi 

1 0 m 0 

2 — Ctt mM, —ca(Mt + M) 
3 0 —1 0 

4 ty Zo — Wy mM, + Catita 


Here co, Mı, Ma are constants. As usual, the first two columns are 
obtained from the KE and the y, are then computed. 
From the first and third columns we observe at once a result of the form 


XVa + avg = 0 
Yara + Vava = 0. 


Then either the determinant here is zero or 73 = v4 = 0. 

The latter alternative requires also that, say, v; = 1, 3 = m,. As con- 
stant v; lead to null A,, our general criterion is trivially satisfied by all 
surfaces. These values of the normals themselves, pertaining only to the 
surfaces of constant V, tell us nothing. 

As for the first alternative: 


Ux. MX. + Mx 

1°2 12 201 2 2 0 

= iC — Mz) = Vz 
1 2 2. 








By My + Cata 





[7.10] A TEST FOR A VOID 187 


The problem to which we will apply these results outlaws x, = 0 so that 
we are left with the surface 





za = V macy, (7.9.23) 


As will be seen later, this just fits the requirements. 


710. A TEST FOR A VOID AND A FURTHER 
NECESSARY CONDITION FOR A 
UNIVERSAL SURFACE 


We continue with games having KE (7.4.1). The initial conditions are 
given on a surface @ which may be the terminal surface or some other that 
arises during solution to play a similar role. Suppose there is a curve 
X on © at which ¢ changes value, that is, A must change sign at %, if 
not on &, then at least in & near 6.12 

There will be a family of paths emanating from @ on each side of X and, 
generally, one of two things will occur. Either the two families will 
mutually intersect and thereby induce a dispersal surface, as was studied 
in Chapter 6 (see Figure 6.5.1), or they will diverge and leave a void, free 
of paths between them, indicative of a universal surface (see Figure 7.5.1).28 
We seek here a criterion to distinguish these two cases. 

We define 


D= 2 (Vises — tisy V; (7.10.1) 
where y; = Oy,/Ox;. ' 


THEOREM 7.10.1. Under the above circumstances, if V exists in & near 
@ and has continuous partials, and the paths leave a void, then at # 


D<o. (7.10.2) 


Proof. In & near @, A is a continuous function which changes sign at 
some surface passing through % and so is decreasing as x passes through 
this surface in the direction from the side where ¢ = —1 to that where 
$ = +1. If the paths from these two sides leave a void, then, at X, A 
must increase more rapidly with 7 on the former side than on the latter. 

These rates of increase cannot be compared through the use of 4, for 
(7.4.6) evaluates this quantity here also and shows it independent of ¢. 

We must look to £ which, after a calculation along the same lines, 
turns out to be Dé + E, where D is given (7.10.1) and £ is an expression 
of similar structure. As A must be greater (or at least equal) when 6 = —1 
than when ¢ = +1, we have (7.10.2). 

12 In Example 7.5.1, if @ is taken as y = 0, then one can verify that A = O and ¢ is 


determined by sgn 4. 
13 Of course, an intermediate case is also possible. 
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COROLLARY 7.10.1. The condition (7.10.2) must hold everywhere on a 2 


¢-US with linear vectograms. 

For we may cut the US with an arbitrary smooth surface and, using the 
values of V on it as initial conditions, let it play the role of @ above, 
because the solution on the far side will agree with the original. The 
intersection with the US will be %; (7.10.2) holds there; hence at all of 
the US. 

Of course, should D = 0, we could proceed with higher derivatives of 
A. But we shall not pursue this subject. It is beset with the classical 
tribulations of attempting sharply to prescribe conditions for a minimum 
in terms of derivatives. When working individual problems, we can take 
D < 0 as a strong criterion for a US and expect that its negation, D > 0, 
on @ will indicate a dispersal surface. A typical instance is 


Exercise 7.10.1 Show that in Example 7.5.1 
D = uu,, 
so that ridges in Figure 7.3.5 are eligible to be US but not the valleys, 


7.11. TEST FOR A TRANSITION SURFACE 


Although this topic in substance is not germane to the present chapter, 


its formal aspect is related. 
Suppose that, when dealing with the integrals of the RPE, in some stage 


of the solution of a game of the type of the preceding sections, we encounter 
a surface S on which A = Sa,V; = 0. That is, on each path, we find a 


certain value of 7 such A(z) = 0, and the totality of the points at which 


this occurs comprises our surface. 
Under what conditions are we justified in asserting that F is a transition 
surface, that is, one at which ¢ changes from one of its extreme values to 


the other? 
As we proceed along an optimal path, by the same reasoning as earlier, 


A = => ViVi 
i 
and is independent of ¢. Suppose at F, A #0. AsA=0 there, then A 


certainly changes sign upon crossing S. Therefore it cannot be that ¢ 
remains unchanged, and so J is a transition surface. Thus we can state 


THEOREM 7.11.1. When the integration of the RPE leads to a surface 
crossed by the paths and at which 


È xV; = YB; = 0, D yV #0 
i 7 i 


then this surface is a transition surface. 
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The utility of this result: Suppose that we have analyzed a game 
according to the ideas of this chapter and have detected all surfaces at 
which 

D V, = > BY, =D vV,=0. 


Then should we encounter as above a surface on which A = 0 but which 
does not belong to this class, we know at once it is a transition surface. 


7.12. FURTHER DISCUSSION OF THE BASIC 
NATURE OF UNIVERSAL SURFACES AND 
THEIR RELATION TO THE EULER EQUATION 


We will erect our ideas around a generalization of Example 7.9.1 and 
Exercise 7.9.2, but will adopt a broader point of view. We consider first 
the general. 


Problem I. The point x moves in the plane (coordinates = %, Xa) with a 
speed w(2,, x2) which is a function of position. To find the paths which 
enable it to reach a given curve @ in the least time. 
Thus the KE are 
% = w cos $ 


č = w sin ġ 


and G = 1. 
Problem II. The same, except that the curvature of x’s path is now 
bounded by 1/R. 


The KE here are 
$ = W COS £3 


Ža = wsin zs 
% = wW(—¢/R) —l<¢<l 
and similarly G = 1. 


The assertion we wish to make here, special instances of which have 
been encountered earlier, is 


The universal surfaces of Problem II are optimal paths of Problem I. 
The proof rests in part on 


Exercise 7.12.1. After adjoining #, = 1 to the KE of Problem II, show 
that the C, yield 


W — = We COS £3 — W SiN Z3 (7.12.1) 


5e 


(Here w; = dw/dz,). 
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We will make an assumption whose strength is between that of the 
F minimax (Section 2.4) and a demand that the KE be separable in all 
= control variables. 


Another part depends on the less routine 


Exercise 7.12.2. For Problem I along an optimal path show that 


Fi = w cos ¢ — w, sin ¢. (7.12.2) ` The a, are free of the dy, Yı (7.13.1) 
That is, œ; is at most a functi f ; : i 
This can be done by our standard procedure, which furnishes first $ as Ur a ost a function of the x,, whereas 8, may involve 


a function of the x, and V, and, secondly, supplies %, and V, (the RPE) as 


Then the general condition for ¢-US z i i 
functions of the same variables, by differentiating the former functions, 2 Geren encore since) Mme tolleying 


generalization of (7.4.8): 


But other means are possible, for (7.12.2) is tantamount to a second-order YaV;,=0 (a) 

differential equation in the x, alone and is the equivalent of the Euler ais 

equation for Problem I. min max ¥ B,V;=0 ($) (7.13.2) 
oe wr i ae 


Now if we replace ¢ by the ¢ of (7.12.1) in the KE of Problem II we 
have the differential equations of the paths which comprise the US. The 
third of these, z, = w4/R, shows that 2, satisfies (7.12.2). Thus, through a 
suitable correspondence of initial conditions, the zg of Problem II will be 
identical with the $ of Problem I. 

The interpretation of all this has been foreshadowed. As in Example 
7.5.2, the tributaries to the US, where $ = +1, are sharpest possible 
turns. That is, in Problem II, they correspond to an early maneuver to ` 
obtain quickly a state—attitude and position—from where x can follow 
an optimal trajectory of Problem I. And these positions constitute the US, 

In broader terms, we see that the US of one game may be tantamount to 
the solutions of another of greater dimension. It would be interesting to 
explore the generality of this phenomenon, but even now we can perceive 
why the construction of US constitute a formidable problem. They bear a 
kinship to integrals of the Euler equation, and we should expect a full 
theory to be beset by all the complexities of the calculus of variations. 


2 yV =0. (y) 


Here («) is precisely the same as its earlier counterpart. In (8) the min 
and max are taken in the same sense as has been usual in our theory: 
they determine each ¢, and y, as functions $, and ¢, of the x, and V, as in 
our earlier derivation of the ME; (8) can be written also as 


2 Biber Po £V, =0 
analogously to the ME,. In (y), y; still means 
2 (aib; — Bia) 
3 


but in forming the derivatives, such as Ø; = 08,/0x,, any appearing ¢, 
or , are treated as constants. The latter are then replaced by the above 
$r and #, functions of the x, and V,. 
The proof is substantially the same as before. The general ME is 
min max > (a, + B)V; 
$:$; i 


Ph 
= min HÈ x; V;) + min max > BW, 
¢ Oi Yk 


= min ¢A + min max B = 0. 


7.13. RESTORATION OF THE TOTALITY OF 
CONTROL VARIABLES 


For reasons already explained, we have confined the treatment of US, of 
the linear type, to those entailing a single control variable which has been 
termed ¢. We now revert to the general situation with the full complement 
of them. However, one will still play a chief role, still termed ¢ (or y if it 
is maximizing) and our subject will be a ¢-US. 

We will thus adopt here a deviant from our usual notation: the above 
¢ will be written without a subscript and y, . . . , $p Yi» » -+ » Y, Will denote 
the other control variables. The linearity of KE in ¢ is still essential; 
their form is still 


As in Section 7.4, an interior minimizing ¢ demands that both terms 
vanish ; thus («) and (8). Finally (y) follows as earlier by the demand that 
A be 0 on the CUS. For on the tributary paths we must have our usual 
RPE, which here are 


,=—(a0+ 8), o=—sgnA 
V, = > Vi(a,40 + Bis). 


i 


t = ag + bi 
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In the second line the derivatives with respect to x; are reckoned with the 
¢, and y, behaving as constants. Thus the stage is set for a repeat per- 
formance of the calculation of 4. 

But when we try for a repetition of workable conditions, such as we did 
earlier for n = 3 and 4, we encounter difficulties. Even the former 
simplicity of the n = 3 case is gone, for the equations (7.13.2) are no 
longer linear in the V; But they are still homogeneous, that is, are 
satisfied when all V; = 0. The new criterion will be the condition on the 
æ; which ensures that nonzero V; solutions exist. The ensuing examples 
will display several possibilities. 

Actually many particular problems yield without a panoply of formal 
ideas. It may be that the restored control variables are few in number and 
the structure of the solution is apparent. For instance, if these variables 
are all involved lineally, then each can have but two possible values, and 
which of these is correct may be inferred from direct simple considerations. 

It should be noted that when we derive the ME in our standard manner, 
(8) will still hold. This is because of our assumption (7.13.1). Thus, as the 
V, are continuous across a US, the restored control variables will be also, 


Example 7.13.1. A generalization of Examples 7.3.1 and 7.5.1. We return 
to our former problem with the 45°-triangle vectograms. These belong to 
the minimizing player P. But now we adjoin a maximizing player E who 
will have circular vectograms with radius v(x, %3), a smooth function over 
the plane. The resulting velocity of x is to be the sum of the selections 
from the respective vectograms. The KE then are (compare those of 
Example 7.5.1) 
2, = ġu + v sin y 
—u + v cos y 


Ža 
ts = 1 
—1 <¢$< l; u>0,v>0; assume always v < u. 
Computing the y, and tabulating: 


a, bi Yi 





u vsin y u,(v sin y) + ua(—u + v cos y) — uv, sin Y 
0 —u+vcosy —(—u, + v cos y)u 
0 1 0 


w Nej ™ 


Our equation (f) is here 


max [v(V, sin y + Va cos yp) — uVa + Va] = 0. 
p 
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As usual, putting p = V V + V,2, 


sing = 4, cos p= É 
. P 

and (8) is j 
= vp — uV, + Vz = 0. 

But («) is uV, = 0 and as u > 0, we have V, = 0. Asv < u, x always 
moves downward and so V, > 0. Thus p = V, and G = 0. That is, ona 
US, E always strives to move x vertically upward. 

Thus (y) is 

V(u, — v)u = 0 


and the sought US condition is 
uy — 0 = 0. 


This is a “necessary” condition only, and remarks made earlier on this 
point apply here also. 


Example 7.13.2. The homicidal chauffeur game. We return to Example 
7.5.2. which deals with the car without the pedestrian and adjoin the 
latter. As above, this means that we add to the KE terms corresponding to 
a circular vectogram of radius wą, the speed of the evader, and we write 
once more the KE of the title game. 


; Wi : 
t= — — XO + Wasin y, 
1 R 2$ 2 Y 


i= 
2 = Rb Wi + wcosy 


= 1, —1<4<1, w >w >00. 
The next step is to form the matrix: 
i Xi B; yen 
1 —(w,/R)2, wą sin y —(—w, + w: cos y) 
2 (w,/R)t, —W, + W cos y We sin y 
3 0 1 0. 


From ($) we obtain, just as in the previous example, 
_ . V; 
sin ğ =~,  cosğ = Va (7.13.3) 
P P 


1 We suppress the common factor w,/R;. 
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Then (y) yields 
n(m — We ^) + ¥,(m 4) = Viw, =0 
p p 


so that V, = 0. If also V, = 0, (8) would imply the then absurd con. 


sequence V, = 0; thus V, Æ 0. As (a) is 


£V =0 
we have as a CUS: 
x, = 0. 


This is the same result that we found in Example 7.5.2 and the dis. . 
cussion there largely applies. Optimal play will be of the direct type 


already depicted in Figure 1.5.2a.¥° 


To see that this is so we must study the tributary paths to the US, but we 4 


do not require a full analysis. We are mainly interested in y, E’s optimal 
travel direction. 
Recalling that the ME, is (we return to the x, y symbols) 


T T lvy, — 2V,)$ — WV, + wp + 1° =0 


and, as on the US, A = 0, the bracket vanishes. As x = 0, y > 0 on the 
US, there we have V, = 0. From our standard procedure 


sin ğ = = = 0, cosp= 4 >0 (7.13.4) 
p p 
we have # = 0 (clearly V, > 0 on the US). Consequently, the points of 
the US correspond to P’s traveling straight and E also, fleeing directly 
from P. 
Turning to the tributaries, we find among the RPE 
p= 07 Vy V, = -07 Ve 
where ø = +1 depending on the side of the US. These equations imply 
that the vector V,, V, rotates with angular speed w,/R. But E’s velocity, 
as seen from (7.13.4), lies along this vector and it too so rotates. But such 
is also the rotation of P on his sharp turns, so that E’s velocity does not 
turn relative to P; in the realistic space E’s path is straight. 
Finally, as V, and V, are continuous at the US, so is E’s flight direction. 
It follows that E’s original velocity must lie on the tangent to one of the 


15 In Chapter 10 we shall learn that part of the lower 2,-axis can also be universal. 
But in an inverted sense, as E now pursues P. 
16 In the terminal payoff form, this 1 is replaced by Vs. 
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minimal curvature circles. Thus the direct play, which we discussed in 
Section 1.5, is optimal. 


Research Problem 7.13.1. The asymmetric homicidal chauffeur game. 
Let Wz, the evader’s speed, be, instead of constant, a function u of x, and 
æ ° Show that the condition for a US is now 


Wily + (yey — Ugey Wa? + 22 = 0" (7.13.5) 


The speed of E has been made a function of his coordinates relative to 
P. A fanciful way of realizing this problem is to think of P as a car with 
headlights (whose beams are perhaps asymmetrical) and the chase taking 
place on rough terrain in the dark. The evader’s speed is the higher the 
more heis illuminated. Thus P has something to gain by pointing askew and 
keeping his lights off his quarry. 


Since the equations (7.13.2) are homogeneous in the V, when n = 3, 
(x), being of degree 1, can be used to eliminate one of the V, say Vs, 
from (£) and (y). These two can then each be written as an equation in 


V,/V_. Our workable condition will then be that these equations possess 
a common real root. 


Example 7.13.3. Let the KE be 
È = 139d, + cos py + ayy 
ty = $1 + xy 
%, = a, sin do. 
Trying for a ¢,-US leads to 
as'V,+ V,=0 (a) 
12V,0 + %V,— p = 9 (8) 
Lits V3 
= (4) any, — 2V3) — (oV, + Vz) =0 (y) 


where p =V V + (V3), o = sgn (£V). 

Eliminating p between (£) and (y) gives us an algebraic equation in the 
Ve. Removing the p from ($) by squaring gives us another. We end up 
with three homogeneous algebraic equations: («) and 


Et Va(28£ V1 — x3V5) + a(oV, + V,)? = 0 
x,°(Vi0 + Va)? = y + aV. 
Eliminating the V, as above leads to 
%y[%gt9(2x,?o,V C? — 1 — 2(C? — 1)) + Ca] = 0 


1 Assuming zv, > 0; otherwise the + is changed. 
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where C = a, (0, — x), o, = sgn (2,V3C), an algebraic equation of a, 


CUS. 


Exercise 7.13.1. Show that, in the previous example, the condition for a ; 


y-US is similarly homogeneous and find the ultimate equation. 
Research Problem 7.13.2. Is it always true that (7.13.2) are homogeneou: 
in the V,? 
7.14. SEMIUNIVERSAL SURFACES 

Such are singular surfaces of type (+, u, p), that is, tributary paths 


appear on one side only; parallel ones on the other. We display their 


existence by the following example. 


Example 7.14.1. The gunner and approaching target.12 The KE are 


¢=—1 
m = —y, (0 < y < 1). 


The payoff is integral with G = yg(x); & is the first quadrant: æ > 0, 
m > 0 while @ is its boundary, the two positive half-axes. Further, g(x) 
is positive, smooth, and decreasing. 

The interpretation is of a gunner located at the origin (x = 0), firing at 
an approaching target at x, this coordinate diminishing at a unit rate. At 
any time the gunner has a supply m of ammunition which he is at liberty to 
fire at any rate from zero to unity. The control variable y is this rate; mis 
a state variable. 

It can be shown reasonably” that the kill probability is an increasing 
function of f yg(x) dx, the integral extending over the partie, for a suitable 
function g as above. Thus the above G. 

The optimal strategy could hardly be more obvious: the gunner waits 
until the target is as near as the ammunition allows, that is, until he can 
just sustain full fire throughout the final interval of the target’s approach. 
The optimal paths, then, are as in Figure 7.14.1.° 

Our interest is the semiuniversal surface, the line OA in the figure. 
Tributary paths appear on one side only. 

It is clear that if @ were suitably modified, any of the 45° paths could be 
made to play the semiuniversal role. Note the contrast to US, where only 
a small number of rather special surfaces are usually eligible and they are 


** A simplification of one case of the bomber and battery game in the Appendix, 
Example A5.3. 

19 See Appendix Al. 

2° Compare Figure AS.4. 
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etermined from the KE and are in good part independent of € or &. 
s distinction precludes a clean, sharp theory for semiuniversal surfaces. 


We work, on the same grounds as for US, with a game of terminal payoff 
with n > 3, one control variable ¢, and linear vectograms. That is, as 
before, the KE are 
#=ap+B, —-1<$<l 

For a possible semiuniversal surface we select a region 2 on a smooth 
surface Y comprised of optimal paths emanating from an initial surface 
¢@. On Z, we suppose regular behavior; say $ = —o = — sgn A. We 
wish to know conditions permitting the use of Ž as an initial surface for 
optimal paths with ¢ = +o. For quantities pertinent to these new paths 
we shall use notations V*, A*, etc. 


THEOREM 7.14.1. Sufficient conditions for &, as above, to be a semi- 
universal surface are 

(1) Throughout &, the plane of the vectogram is not tangent to F. 

(2) On 2, sgn > yiV,* = o. (The V,* can then be intrinsically determined 


t 
on & as shown below.) 

Proof. Let us select the parameters s,,...,5,_, of & so that for F N @, 
S,-1 = 0. On this curve let H be (s,,..., 5,9). The parametric equations 
of Z are then 

T; TT u,(7, Say eee, Sno) (7.14.1) 
where the right sides are integrals of the RPE with the usual initial 
conditions, that is, as determined from H on @. These RPE also imply 
that 

E, = —ou, + pi 

On F, 

V = NSi o o y Spa) (7.14.2) 


A 














Figure 7.14.1 
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We are going to use (7.14.1) and (7.14.2) as initial conditions in the 4 


usual way, but for the paths obtained we must have $ = +0. Theg 
customary initial conditions, relevant to the parameters 5,,... 
are then 


s Sno T 


eV 2I Vi" pies. k=1,...,n—2 
0s, 0s, t ds, 
ov 


> =0= > (—oa,; + BV," 
Or i 


and we adjoin the ME for the new system 
0 = J (oa, + BV. 


The last equations are equivalent to 
È aVi(= A*) = 2 BV* = 0, 


the determinant 
Ti 


f: 
in virtue of (1). Thus we can find the V;*. As A* = 0, $ = —sgn A’* if 
the latter Æ 0. But (2) requires that 


o = — sgn A* = sgn > yV," 


and this suffices for at least a local path construction. 
COROLLARY 7.14.1. On a semiuniversal surface, 
A* = B= 0, 


Suppose & is not all of S through failure of the condition (2). Then on 
the boundary of Z, we should expect that 


2 yV; * = 0. 


But this equation and the two of the corollary are just our necessary 
conditions for a US, implying that one may emanate from the boundary 
of Æ. For a typical instance of such a possibility, see Figure 7.14.2. 


(7.14.3) 


SEMIUNIVERSAL SURFACES 199 


US 


R 


Figure 7.14.2 


Here S is AC, but Z is BC where (2) holds. At B, then, we have (7.14.3), 
The US is curve BD. Its tributaries on the lower side merge with those of 
#; those above the US are an extension of family of paths on the far 
side of F. 

Observe that if (1) and (2) hold on a surface, they hold on neighboring 
surfaces. Thus candidates for semiuniversal surfaces occur in families. 

It is hard to say much more. Of the abundance of candidates, the few, 
if any, will depend on @ and &. As in our example, such a one must be the 
extreme of a family bounding a void (or “‘semivoid”). 

The wonder is, that considering their profilic possibilities, semiuniversal 
surfaces have appeared much more rarely than US in the examples we 
have considered, both in and out of this book. 

An interesting possibility appears in Example A4 of the Appendix. 
There @ is at first.a circle and two transition curves emanate from it. As 
the radius becomes zero, each curve becomes semiuniversal. 


Problem 7.14.1. Taking the KE of Example 7.14.1 


z = —İ 
Ly = —y 
t = pga) 


let @ be xz, = 0; 6,2, >0. Take H = x, and suppose that g(r) > 0 for 
0 <7 < To but g(7) = 0. Take F as a family of optimal paths through 
the curve in @: x, =0,2,=5, 2, = J(s) where f(s) > 0. Show that S 
is a possible semiuniversal surface as long as T < To 















































CHAPTER 8 
Games of Kind 


Typical of a game of kind, in distinction to one of degree, is a pursuit 
game in which we are interested in what conditions make capture possible 
for the pursuer or escape for the evader, rather than seeking the best 
procedures in terms of optimizing some continuous payoff. 

Until now there was, as far as we know, only one approach to such 
problems. We have dubbed it the Method of the Explicit Policy. It 
consists of, say, demonstrating the possibility of capture by exhibiting a 
particular strategy or policy of the pursuer which attains it. Its weakness 
is that, in almost all positions, there is no determinate best decision for 
either player; to seek a sequence is to grope in the dark. One can seldom 
solve all cases of one particular problem, let alone solve all problems of a 
class. 

The innovation here emphasizes the hypersurface, called the barrier, 
which separates, in the space of starting positions, those of capture from 
those of escape. For starting points on the barrier, optimal behavior leads 
to a contact of the terminal surface without a penetration. We term such 
an outcome neutral and regard it as intermediate between capture and 
escape. The advantage is that for neutral outcomes alone do there exist 
determinate optimal strategies. 

Differential equation techniques, similar to those of games of degree, 
lead to optimal strategies and paths and thence to the barrier. The 
global answer to capture-or-escape question then hinges on whether or not 
the barrier divides the playing space into two parts. 

One of the chief difficulties is the finding of the proper initial conditions 
or the means of attaching the barrier to the terminal surface. We have 
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discovered three, all rather different, one in particular being tantalizingly 
subtle. Between them they cover all practical instances thus far encountered 
but do not appear to be exhaustive. 

Significant examples of problems solved by these methods will appear 
in the following chapter. Space here has limited us to simple illustrative 
instances. 


8.1. INTRODUCTION 


In games of degree, we recall, the players strive to maximize and 
minimize a certain payoff capable of assuming a continuum of numerical 
values. For any partie, the particular value of the payoff is not ascertained 
until termination, that is when x reaches the terminal surface €. Thus 
our whole theory of such games is erected on the presumption that @ is 
actually reached. 

In this chapter we discuss games of kind, in which the achievement of 
termination itself is the quintessence of the problem. 

We shall almost always suppose that one player wishes termination and 
his opponent does not: these opposing desiderata are the conflict of the 
game. But other circumstances are possible and indeed useful. For 
example, collision avoidance between two moving craft.1 Both pilots 
wish to Prevent “termination” (= collision in our format). Although such 
a situation is not a differential game, it is amenable to the same techniques. 

We shall use the vernacular of pursuit games as a surrogate for all cases. 
That is, we will assume that it is P who desires termination—capture in a 
pursuit game—and Æ, its avoidance. 

To subsume these games technically into our general scheme we can 
assign numerical values to the outcomes, retaining the concept of the 
payoff. For example, 


+1 for no termination or Escape 
—1 for termination or Capture 


enable P and E to remain the minimizing and maximizing players. 
With such payoffs, the general theory of games defines the Value as 


+1 if there exist a strategy for E such that, when he plays it, termination 
will never occur no matter what the strategy of P. (8.1.1) 


—1 if there exist a strategy for P such that, if he plays it, termination is 
certain to occur no matter what the strategy of E. (8.1.2) 


* To appear at a later date. 
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Although we shall not require the formalism of numerical payoffs in 
practice, note that it is consistent with the general nomenclature to define 
the two particular strategies mentioned as optimal strategies for E and P. 

In contrast to games of degree, in most cogent examples the optimal 
strategies are not unique. Indeed they are legion. Consider, for example, 
any reasonably simple pursuit game in which P is so kinematically superior 
to E that he can capture from any starting position (V(x) = —1 for all 
xe). As we are interested only in capture at some time, P may loiter as 
long as he likes; there are never preferred values of the control variables, 
Similarly, if the superiority attaches to E, so that escape is always possible, 
E has complete freedom of action save possibly when capture is imminent 
and even then he may evade with as close a shave as he pleases. 

To prove that P can capture or E escape, it suffices to exhibit one par- 
ticular strategy that will enable him to do so despite any opposition. This 
most obvious technique for games of kind we shall call the Method of the 
Explicit Policy.? An instance is: 


Example 8.1.1. The game of two cars. The points P and E move in a 
plane each at a fixed (or bounded) speed and each with its curvature 
bounded. The four bounds may be any magnitudes. Capture, as usual, 
means |PE| < /. Under what conditions can P capture £E? 

If P has the higher speed and at least as favorable a curvature restriction 
as E, capture can be attained. For P can first go to E’s starting position and 
then follow his track. 


Generally such a display of a particular policy suffers the drawback of 
there being no systematic way to find one. We have to be ingenious anew 
in each case. For example, it seems reasonable that P could capture with 
a slight inferiority of curvature provided he was sufficiently superior in 
speed and / is generously large. How do we exhibit a policy to prove it? 

We offer in this chapter what appears to be an approach of greater 
generality, with wide, if not universal, applicability. By a slight but natural 
alteration of the criterion, we incur a subset of & on which optimal strate- 
gies are determinate if not unique. Artificial, inefficient policies, as that of 
P above, are avoided. 


8.2. THE BARRIER CONCEPT 


First we modify, for a definite purpose, the definition of termination. 
We demand that x not only reach @, but also penetrate it. Should x 


2 As the ensuing example will show, it may be convenient to admit playing policies 
which are not strategies. 
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reach @ without crossing it and ultimately return into &, the outcome will 
be considered as neither capture nor escape but as a third alternative. 
Numerically we can say that now 


payoff = 0. 


We will refer to this third possible outcome as neutral. It is to be 
regarded as the delineating case between capture and escape. Our motive 
is that only in cases of neutrality (V = 0) is each player’s actions, as he is 
on the verge of a worse payoff throughout play, decisive. Generally it is 
in this case alone that there exist definite and determined optimal strategies. 
Here there is material for a calculable theory. 

We recall that a differential game is really a family of games, one for 
each starting position. One of three possibilities must occur: 


(E) (8.1.1) holds for all starting points of &. 
(C) (8.1.2) holds for all starting points of &. 
(M) @ contains both kinds of starting points. 


The set of starting points for which (8.1.1) holds we will call the escape 
zone (EZ); those for which (8.1.2), the capture zone (CZ). 

When (M) prevails, the two zones will each be a region. Generally they 
will be separated by a surface which consists of the starting points for which 
the outcome is neutral. This surface will be called the barrier. 

The nucleus of our approach will be the ascertainment of the barrier. 
Knowledge of it will automatically engender that of the CZ and EZ and so 
distinguish the situations from which P can force capture or E escape 
whenever (M) obtains. And the latter happens in the most interesting 
cases, for when (E) or (C) holds, the situation is often one sided enough to 
be transparent. 

Even when (M) does not obtain, our ideas can apply. For the game will 
always entail parameters—speeds, capture radius, etc.—which appear 
formally in the. coefficients in the KE, H or G or specification of @. By 
varying them we can often occasion (M) when it did not pertain originally. 
Thus we imbed the game in a continuum by varying some or all of the 
parameters. 

For example, we have seen an instance of the game of two cars in 
which (C) holds (the capture zone = &). Now suppose E has the greater 
speed. If initially Æ is pointed away from P, he can escape by going 
straight. But for a starting position in which P and E point toward one 
another and are sufficiently close, it is intuitively clear that capture ensues. 
Thus both zones are not vacuous. There will be a surface Z, the barrier, 
separating them. 

Now starting from such a case, let us continuously vary the parameters 









































204 GAMES OF KIND [8.2] 3 


in some way that tends to favor P. We know that this can be done so that 
at some point & becomes all capture zone. What happens to the barrier 
then, which we suppose has deformed continuously? It will, at some 
critical point, have ceased dividing & into two parts. 


This situation is general. We shall extend the term barrier to include © 
surfaces which do not divide @ into two parts, when such a surface ~ 


derives from the barrier (in the original sense) through a continuous 
change in the parameters. This definition may appear vague; what we 
have in mind is the result of a construction which will be given shortly. 
The barrier in this extended sense—when it does not necessarily delineate 
the capture and escape zones—still constitutes, as we shall see, an im- 
portant singular surface when certain basic continuous payoffs are adopted 


and the game becomes one of degree. Generally the barrier is then of type - 


(p, —) or (—, —), is not crossed during optimal play, and marks a dis- 
continuity in V. Although it may not bound an escape zone, it does 
delineate a region in which P’s task is more difficult. 

Inasmuch as there will be determinate best ¢ and y only when the out- 
come is neutral, that is, x is on the barrier, we shall use the term optimal 
strategy when dealing with games of kind in this limited way. That is, 
(x) and #(x) are now defined only for x of the barrier. 

Perhaps some will find the distinction of the neutral outcome objection- 
able. As @ is a closed set of &, any meeting of x with @ should be capture. 
Penetration of @ is meaningless in that the no KE are postulated for x 
outside of &. 

These objections can be assuaged by the following modification of 
concept which permits retention of the convenient barrier methodology. 
Let us think of a surface @, parallel to @, a distance e away, and within &. 
We shall assume that the continuity of the KE causes all barrier con- 
structions based on @,, to be near those of @ if € is small. Let E, starting 
from some x of the escape zone, pick ¢ so small that x is in the escape zone 
relative to @,; then let him play as if @, were the terminal surface. His 
strategy will be arbitrary unless x is on the new barrier; there he plays 
optimally. But a neutral outcome in the new game means escape in the old. 
If x is in the capture zone, P plays optimally as in the original game. 

Very basic is the fact that 


The barrier is a semipermeable surface. 


For let x be on the barrier separating the escape and capture zones. 
Now P must be able to select a value ¢ of ¢ which prevents x from entering 
the escape zone, for otherwise E could attain escape starting from x. 
Similarly, E can keep x out of the capture zone. These abilities to prevent 
penetration comprise the definition of a semipermeable surface. 
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Note that, in addition, a particular orientation of this surface is required; 
namely, the penetration direction which Æ can prevent must lead into the 
capture zone. 

Thus we must next turn to 


8.3. THE CONSTRUCTION OF SEMIPERMEABLE 
SURFACES 


Let F be a smooth surface in £, and at each of its points let » = 
(v4, +++» Yn) be its normal vector. That is, each v, is a function of the z, 
defined when x € S. The length of v, as long as it is not zero, is arbitrary; 
it can be taken as any convenient nonzero function of x when working 
problems. But the orientation of v is important. Thus v is determined up 
to multiplication by an arbitrary positive function of x. 

The condition that be a semipermeable surface 


m max 2% Fx, $, y) = 0. (8.3.1) 
y i= 


This equation can be regarded as our usual ME with a change of 
interpretation. Indeed (8.3.1) will be referred to as the main equation. 
However, we shall give an independent derivation. 

As fi is #,, the ith component of the velocity of x when ¢ and y are 
played, the sum in (8.3.1) is the component of this velocity in the direction 
of y. Then this sum’s being >0 [<0] is equivalent to x’s penetration of 
S $ the [opposite] direction of ». Now let ¢ supply the min in (8.3.1); 
we have 


0= max 2 v(x, $, y) > > vf, $, y) 


for any y on the extreme right. Thus the use of ¢ assures no penetration in 
the v direttion, for any y. Putting the same shoe on the y foot concludes 
the proof. 

We select a definite $ and @ for each point of which furnishes the 
min and max in (8.3.1). Furthermore, we suppose them reasonably 
smooth functions over F. Then at each point of F, x will have a definite 
velocity {f,(x, $, )}. 

It may be that this velocity is everywhere zero on Z. In this case we 
shall call the semipermeable surface static. The equivalent formal 
condition is obviously 


S69) =0, i=l,...,0, (8.3.2) 


Example 8.3.1. Let n > 1 and P and E each choose unrestrictedly the 
direction of a unit velocity vector, the net velocity of x being the vector 
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sum of these two choices. Then, obviously, any smooth surface is semj4 


permeable. The players choose velocities lying along v» but in oppos 
directions. It follows that the surface is static. 


But the case where (8.3.2) nowhere holds is much the more interesting, 
Assuming a nonzero velocity at each point of S, from (8.3.1) it is clea 
that this velocity will be tangent to Y there. From the existence theorem 
for differential equations, we see that / will thus be a union of paths such 
as are described by x when governed by the strategies $ and ¢. 


On the assumption that Z can be imbedded in a family of semipermeable: 
surfaces which univalently fill a neighborhood M of it, we can derive a set: 


of RPE for these paths. The derivation contains no formal novelty. It is 
the same as was done before with the v, replacing the V,. 
Explicitly, the required assumption is that y can be extended throughout 


M. Choose particular smooth ¢(x, v), Ẹ(x, v) satisfying (8.3.1) and 


defined for xe M. Write (8.3.1) in the “ME, form” 
> yf, $, p) = 0. 


i 


Then let v, be functions on / satisfying (8.3.3) there and equaling the : 


original v, on S. Such is the needed extension. 


Let $ and #, functions of x,, v; furnish the min and max in (8.3.1). We 


differentiate > »,f,(x, $, P) with respect to x, and consider the different type 


a 
terms. First, those due to the externally appearing v; give, where v, means 


Ov,/Ox;: | 

> SAX, $, P). (8.3.4) 

The terms due to the explicitly appearing z; in the f; are 
2 vfl, $, P). 


Next, those x; appearing as arguments in 4(z,, »,(x,)) which are 


2f. $y 
ers an, 


(8.3.5) 


which can also be written as 

>(2 zos) (2). 

Epe T I \Ox, 
But (as we reasoned once before), if $, is an interior minimum, the first 
parenthesis vanishes. If it is an exterior one, $, remains for all local x at 
one of its extreme limits, which we may suppose to be constants. Then the 
second parenthesis is zero. 
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Similarly the terms entailing the arguments of @ vanish in any case. 
Classical methods of analysis can be used to show the existence of 
` F(x) defined on / such that, if the v; are of proper length, 


OF 
v = —. 
GEA 
This implies 
d» _ 2 
Ox, On, 
OF Mig = Viw 
We make this replacement in (8.3.4), which becomes 
: 2 Vifi 
© which is also 
Ov, dx; 
2 Ox, dt 


the latter derivative pertaining to the motion of x when A(x, »(x)) and @ 
are employed as strategies. But the last form of (8.3.4) can be written as 
y» Thus we are led to 
Pym a > Sis: 
t 


Finally, we take this equation and the original KE with ¢ and y re- 
placed by $ and @ and reverse the time direction (r replaces t) in both: 


a, = —f,(x, $, Pp), ”; = x vifaa, $, Pp) 
our new RPE.? 


Research Problem 8.3.1. Can the RPE (8.3.6) be derived without utilizing 
the neighborhood VY of S? 

Although this problem is a natural one at this point, its practical 
consequence is not great for us. To apply our ideas to games of kind, we 
shall find the membership of in a family useful. 


(8.3.6) 


In general,* we can pass a unique semipermeable surface (with a proper 
orientation) through a given curve (= (n — 2)-dimensional manifold in £). 


Let the latter be 
D: x; = hfs, ..., Sn-2) (8.3.7) 
? We shall always use the numerical subscript notation for the »,. 
t The scope of the “in general’ is like that in the familiar theory of first-order partial 
differential equations. Any curve will serve as a proper seat of initial conditions if it is 
nowhere tangent to a characteristic. 
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We must first fit the v; to 2. Normality requires that 


S hy = 0 (w=), j=1,...,n—2 (8.3.8) 
i S 


3 


and the ME, (8.3.3) is also to be satisfied on 2. 
Thus we have n — 1 equations to determine the v; One must note 
whether the orientation of v fits the problem at hand, but otherwise the 


length of v is arbitrary. 
The following result is a sort of converse, showing that our construction 


leads to a semipermeable surface. 


THEOREM 8.3.1. Let $ and @ denote functions of the x; and v; which 
furnish the min and max in (8.3.1). For a curve 2 given by (8.3.7) let 4, 
be values of v; not all zero satisfying (8.3.8) and (8.3.3). Let 2(7,51,..., 
Sne) and ¥,(T, S . » - , Sng) be integrals of the differential equations (8.3.6) 
with h, and 4, as initial conditions. Then (7, S}, . . . Sn—2) is the parametric 
representation of a semipermeable surface which contains 2. 


Proof. Let Q = > vf x, $, p), and let us calculate Ô, the (retrograde) 
i 
rate of change of Q along each path [as expressed by the x;(7)]: 


ô = Fo + ZSP 


Now f= Efst- $= — Z fafs 
The dots stand for terms such as 

f: Od, 

O¢, Ox, 


which, as discussed earlier, are all zero. The second equality results from 
the first of (8.3.6). Thus the first sum of Q is 


T D firf. r 
ij 


This is annulled by the second sum, if we replace v, by its value from 
(8.3.6). Thus 6 equals zero. Because Q equals zero on 2, it equals zero 
on F and (8.3.1) is satisfied there. 

Similarly, if 
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then 
> a a 
R= - 392+ (3r) 
T OS, F 0s, 


of, (X, $, p) 
0s, 


This yields R, equals zero after substitution and an interchange of i, j 
in one of the two sums. Because R, equals zero on 2, due to (8.3.8), 
R, equals zero on P. 

These n — 1 equations and (8.3.3) holding on S imply that v is normal 
to S. Then (8.3.1) implies that S is semipermeable. Obviously the points 
of F with r = 0 form 2. 


The proof also shows that 

If $ and # as determined from (8.3.1) are unique, there is a unique 
solution for every essentially distinct v, satisfying (8.3.8) and (8.3.3). 

The semipermeable surface obtained by the foregoing procedure is 
imbeddable in a neighboring family of such surfaces. It is only necessary 
to imbed the curve 2 in a one-parameter family of nearby curves and use 
each. The family will constitute a surface and it is, of course, necessary 
that this surface is not tangent to /. 

We have been assuming that YS is smooth. The scope of our work can 
be broadened to include various types of exceptions. The ideas are as in 
our previous pages. For let S be imbedded in a family of semipermeable 
surfaces. Let us select a smooth function F which is constant on each but 
with a nonzero rate of change throughout the family. Clearly what we now 
have is tantamount to the surfaces of constant Value (V = F) of a terminal 
payoff game. To it we may apply our earlier concepts of singular— 
transition, universal, etc—surfaces. These are reflected as corresponding 
singular curves on S. 


Exercise 8.3.1. Given the KE 


= DSi oa + terms that = 0 as before. 


Si 


and ` 


t=l—d—%, -l<¢<b 
pass a semipermeable surface through the x-axis (parameterized x, = 0, 
Za = S, % = 0). Two are possible of opposite orientation; settle the 
choice by assuming v, > 0 (one may take v = 1). We should have v, < 0 
except on J; why? 
5 As long as v; < 0, it is not necessary to place bounds on y, as will appear from the 
calculations. 
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[The solution is 
x, = 4st — 47% 


Z = S — 27? 
x; = 287? — 74.) 
Exercise 8.3.2. For the KE 
£= cos $ 
ý = sind + 2y, -l<ey<l 
1. show analytically that the semipermeable surfaces are lines making a 


30° angle with the vertical and find ¢ and 9. 
2. by drawing vectograms, demonstrate this result geometrically. 


8.4. TERMINATION OF BARRIERS 


It is quite possible for a semipermeable surface to come to an abrupt 
end, none of the paths being continuable beyond a certain curve In the 


surface so that the latter remains semipermeable. 
That this phenomenon can occur is shown by three examples: 


Example 8.4.1. Let the KE be 


¢=¢(1—y)—(1—-y) 
y=—chte—1, (c=aconstant; —1 < $ < 1) 


and we shall pass on semipermeable surface through (0, 0). 
The ME is 
min {$b — y) — rec] — 7 — y) + v(e — Il} =0 


so that 
= —o = —sgn[ ]= —sgn A. 


The RPE are i 
g=(¢+11-y), %=0 
g=—(+et+1, n= (0 + ij 
As initial conditions we shall take », = 1, v = 0, as well as x = 0, 
y = 0. Such is consistent, for here 
Az=1(1—0)—0c=1>0 and soo = —1,¢=1 


and the ME is satisfied. 
As the RPE here are 
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they have the integrals 
= 0, n=l 
=7, n=0 
and d = 1—7. 
Thus A > 0 until r = 1 and thus far the path is the vertical segment 
extending from (0, 0) to (0, 1). Here A changes sign and a continuation 
would have to have o = +1, rendering the KE 


(8.4.1) 


Now if c > $ then ġ = 1 — 2c < 0 and the new path, if there is such, 
starts out downward, doubling back on the old. As the normal v does not 
change at the junction, while the direction of traverse does, it follows that 
the orientation changes, and there can be no semipermeable continuation. 

On the other hand, if c < 4, the integrals of (8.4.1) present a perfectly 
valid continuation of our vertical segment. It is, as the reader may easily 
verify, 

æ = (1 — 2c)7? 


=1+(1—2c)r ° 
an arc of a parabola joining smoothly to the top of the vertical segment 


and forming with it a valid semipermeable surface. 
It is instructive to work 


Exercise 8.4.1. By drawing vectograms etc. interpret the above example 
geometrically. 


A still simpler possibility is demonstrated by 
Example 8.4.2. The KE being 


%= sind 
ý = cos  — q(x, y) 


the vectograms appear as in Figure 8.4.1. If g > 1, either of the over- 
scored arrows (which one is a matter of orientation) are tangent to a 
semipermeable surface, as all the other arrows lie on the same side of 
these. On the other hand, if q < 1, clearly none can exist locally. 

Let q, a function of x, y, be >1 in part of the plane and <1 in a second 
part. In the first part we can draw semipermeable surfaces. Should one 
of these tend to enter the second part, it must immediately terminate. 


* Not the old 7; this one starts anew at the extension. 






























































212 GAMES OF KIND [8.4] 


3 
| 


q>1 q<l 
Figure 8.4.1 


Exercise 8.4.2. By setting up the ME, RPE, etc., verify the preceding 
conclusions analytically. Construct and solve a specific case. 


Finally, we display a typical instance of a proper two-person game. It 
will reappear later as a constituent of a broader matter in Chapter 10. 


Example 8.4.3. Take & as the upper half-plane (y > 0) and let P have 
the vectogram XA,A, shown at (a) of Figure 8.4.2. The headline 4,4, is 
vertical and its half height 4,A, or A4 is the constant b while the 
horizontal component XA; is u(y). The circular vectogram of constant 
radius w belongs to E. The two velocities are additive so that the KE are 


& = u(y) + w sin y 
ý = —b¢d + woos y, -l<¢<l. 


Here u(y) is positive, smooth, and increasing for y > 0. For some Yo, 
w = u(y) so that w > win and only in the strip0 < y < yp. Also b > w. 

We will pass a semipermeable surface through the origin with the escape 
zone lying to the left. We shall soon see that it appears as at (b) of the 
figure, extending from 0 to a point B where «(= xg) is positive and 
y (= YB) = Yo 

When yY < Yo, draw the circle of center A, and radius w as shown at (c). 
We will demonstrate that the tangent XT, drawn as shown to this circle is 
the needed semipermeable direction at X. First let P play XA, ($ = 1). 
Then the resultant velocities at E’s disposal all begin at X and terminate 
on the circle; clearly none penetrates XT, in the direction of the arrow. 
On the other hand, if E plays A,7,, P can but choose velocity vectors 
extending from X to T,T, (here A,7, is a translate of A,7,) and none 
penetrates in the reverse direction. Hence the semipermeability. 

Now suppose y > Yọ so that u > w. Then (see (d)) the line 7,7; lies 
on the same side of XT, as the circle and the semipermeability fails. In 
fact, no such direction exists with the proper orientation. 
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In the strip 0 < y < yy we have an XT, through each point X and so a 
direction field. The classical theory of differential equations permits us to 
draw a curve through 0 having the local XT, as tangent at each of its 
points; its form is as at (b). Note that at B, the tangent is vertical, an 
obvious conclusion from the vector diagram (c). 


Problem 8.4.1. Treat this example analytically and corroborate the fore- 
going results by the methods of Section 8.3. 


Exercise 8.4.3. Show that the differential equation satisfied by the 
barrier is forO < y <S Yo 


d re 
(b — w*) T = w/ u(y) + b? — w? — bu(y) (8.4.2) 
=-1 
Al Tı Ay A 1 
| a Ti 
b b 
A 
X 
b b 
ae 
ALK \ 
o= +41 
uly) > T A \ (d) 
(a) (c) Az 
Te 
y 
B 
(0) 
x 
Oo 
Figure 8.4.2 
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and obtain the barrier for the case? where 


b=3, w=2, u=ytl. 


so that Yo = 1. 
[(8.4.2) can be derived easily by observing that normality requires 
ARE d. 
dy OA 


and the ratio on the right can be found directly from the ME;.] 


8.5. CONSTRUCTION OF THE BARRIER 


Our approach will be toward the delineation of the escape and capture 
zones, when both exist, by investigating the surface—the barrier—that 
separates them. We know it must be semipermeable. In the last section 
we saw how to construct such through a given curve. We now face the 
problem of ascertaining which curve. 

From (8.3.1) we see that the normal vector v to the barrier should 
extend into the escape zone. We will always adopt this orientation 
convention. a 

In many important cases the barrier will meet @ and the initial curve can 
be taken as the intersection. This idea of taking 2 on @ isin accord with 
our general scheme of beginning on @ and working retrogressively into &. 
That matters need not always be so is shown by 


Example 8.5.1. We take as KE 
£ = cos $ + u cos y 
j = sin $ + usin y 


where u = u(x, y) is a continuous function enjoying the properties: 
For y >0,u>0. Letting F bea smooth curve in the upper half-plane 
(y > 0) which meets each vertical line exactly once. Above F, u> 1; 
on F, u = 1; below F, u < 1. 

We take & to be the upper half-plane (where y > 0) and @ to be the 
x-axis. 

It is clear that the velocity of x is vectorially the sum of two velocities, of 
magnitudes u and 1, with their respective directions under control of the 
players. Clearly, also, above S, where u > 1, E controls the direction of 
motion and below Z, P does. T hen X is the barrier. On it the two 
velocities each have magnitude 1; the players pull against one another, 
oppositely on the normal to SF. Consequently, F is static. Clearly the 
capture zone is the subset of below F and the escape is similarly above. 


1 We will utilize this case later. 
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Research Problem 8.5.1. The same as the last example except that a 
constant C is added to the expression for #. If C is sufficiently small the 
barrier can be expected to be near S. Is it still static? 


From now on we assume 2 to lie on @ and note three distinct possibilities. 

I. Natural Barriers. Here 2 will be the boundary of the usable part. 
In the light of the present subject it is instructive to review the latter 
concept. 

Let x be a point of @ and let y = (71, ..-, Yn) be a nonzero vector at x, 
normal to @ and extending into &. The condition 


min max > ,yf(x,¢,y)<0, xE (8.5.1) 
$ p i 


expresses the fact that P can force x to penetrate @ despite all opposition. 
The subset of ¢ for which (8.5.1) is true is the useable part (of @). If we 
replace < by >, then Æ can frustrate penetration and such points con- 
stitutes the nonuseable part. The delineation—the BUP—is characterized 


b 
y min max > y,f(x,¢,y)=0, xe@. (8.5.2) 
¢ v i 


For such points, when each player exerts his optimal endeavor x moves 
(if at all) tangentially to @. 

As the BUP separates the points on @ (or rather a little away from 
G) where immediate capture will ensue from those with immediate 
escape, it seems logical to use the BUP as the initial curve 2 for the 
barrier Z. 

To construct # we use for initial conditions: for x, the x of the BUP and 
for v, we use y. As y is normal to €, and coincides with y where Z meets 
@, the two surfaces meet tangentially. An ideal prototype is sketched in 
Figure 8.5:1a. The BUP here is depicted as a closed curve on @, the 
useable part being its interior. The barrier meets it tangentially to @ and 
appears here as a trumpet shaped surface. It is the union of paths, whose 
directions are shown by arrows which meet @ tangentially at the BUP and, 
as must be true generally, from the useable side. 

Suppose now that Æ actually separates & into two parts. If x is in the 
outer side—the one not contiguous to the useable part——then P cannot 
compel capture. For he can force x neither through the semipermeable Z 
nor through @ as only the nonuseable part is accessible. 

To show that the inner side of Z is the capture zone is harder to prove. 
At (b) is shown a partial cross section of (a) of the figure, cutting the 
latter ata path. Let be imbedded in a one-sided family of semipermeable 
surfaces as indicated by the dashed curves. These are assumed to bound 
ever smaller subsets of & over the useable part. Now let x be on the inner 
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(b) 
Figure 8.5.1 


side of Z, as X; is in the figure; X, lies on a surface of the family. If E 
plays a # germane to it, P will play $; x stays on the path which leads him 
to and across @. If E plays otherwise, then P can force him to penetrate 
through the family. This penetration is irreversible and—certainly in 
many particular cases at least—x is drawn irrevocably to &. 

Finally, let x be on Z. If $ and @ are played, x traverses a path of Z, 
meets @ tangentially, and leaves it again (the dotted curve of (b)). Thus 
the outcome is neutral. 

A defection by either player will lead to a worsened payoff, that is, 
either definite termination or escape. Thus here the optimal strategies of 
both players are really optimal in the customary sense of this term. Such 
is true nowhere else. For example, when outside of Z, E need adhere to no 
particular y. He can fix on some semipermeable surface neighboring Z 
on the outside and arbitrarily close to it and not act decisively unless x 
reaches this surface. 

We construct Z by starting at the BUP and integrating the RPE 
therefrom. The resulting surface may or may not divide & into two parts. 
In the former case, these will be the sought escape and capture zones as 
just shown. 
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If @ fails to subdivide @, then capture can always be attained by P. 
But from starting points on the (in a local sense) two sides of Z he must 
adopt different tactics. The typical idea is sketched in Figure 8.5.2. From 
Xa we can expect rather a direct path to Y. But from X,, P must force 
(assuming perhaps some reasonable resistance from E) x to follow the 
indirect path which skirts Z to reach @ at a point of the useable part. 
An example is the swerve maneuver in the homicidal chauffeur game, and 
there is a pristine one in the isotropic rocket game (see Figures 5.5.3, 5.5.4, 
and 5.5.5). 

All portions of # need not be relevant to the problem at hand. Should 
Z intersect itself, or consist of several parts which do, the portions 
beyond the intersection are to be discarded. Thus in Figure 8.5.3a, we 
reject the dashed parts of Z; the capture zone is the shaded curvilinear 
triangle. A beautiful instance of this, open to the most natural of inter- 
pretations, appears in the homicidal chauffeur game, to be discussed in 
Chapter 10. 

Figure 8.5.3b depicts a discouraging possibility. Construction of the 
retrograde paths from some points of the BUP such as P,, may be quite 
satisfactory. But from others the paths may dip below @ when near the 
BUP and then rise above it, (P,) piercing the useable part at a point A. 
Clearly such paths cannot fulfill the role we wish of them and must be 
discarded. An instance in the isotropic rocket game was for long the most 
frustrating problem encountered in the theory. 

IJ. Artificial Barriers. Suppose we have a first game for which @ has a 
useable part (which might be the whole of @). The ultimate game is 
modified by adding the rule that termination must occur by x reaching @ 
at a point within some definite subset of this part; this subset being 
bounded by a curve Y. To construct the barrier of the latter game, we 
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Figure 8.5.2 
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(b) 


Figure 8.5.3 


would begin by attempting to pass a semipermeable surface through 2. 
Such is one type of what we mean by an artificial Z. 

Of course, the modification may not be naively stated as a curtailment 
of @ but may be a logical equivalent. Thus if @ were comprised of several 
analytic surfaces, 2 might lie at their intersections. A polygonal ¢ is an 
example, only one face of which turns out to be the useable part. 

A pursuit game with several pursuers hunting a single evader furnishes 
another instance. Another class restores the single pursuer but has the 
evader constrained by certain boundaries or obstacles. We can immediately 
consider the surfaces delineating these as additional surfaces of termination, 
regarding a transgression by x as tantamount to capture. 

In all of these cases we begin by passing a semipermeable surface 
through 2 according to the methods of Section 8.3. Most of the pre- 
ceding ideas of this section hold with evident modifications. 

Ill. Envelope Barriers. Let G be a differential game with n > 3 and 
with a nonempty, nonuseable part of @. The initial curve ((n — 2) mani- 
fold) 2 for this type of barrier lies in this part. The optimal paths 
emanating from 2 meet 2 tangentially (hence the term envelope barrier), 
and the optimal strategies on the barrier are extendable continuously onto 
2. Thus when x follows an optimal path to @, it can continue its motion, 
with both strategies continuous, and travel on 2. 
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Naturally only special 2 are eligible for the role and they do not exist 
in every game. Let x be in the nonuseable part. For any ¢, then, E can 
find a y causing penetration into @; let us suppose he can also find a 
y = P = Vx, $) which keeps x on @, that is, its velocity is tangent to . 
Then when E plays ~, we have a one-player subgame G, whose @ is 
(part of) the nonuseable part of G. The essential requirement on 2 is that 
it be a semipermeable surface of G,. 

We shall elucidate this in a theorem and then explain the role such 
surfaces play as barriers. 


THEOREM 8.5.1. Let G be differential game with n > 3, for which the 
following is true: 

1. There is a region Z of the closed nonuseable part such that for any 
¢* and x e &, there is a value y(¢) (== w(x, ¢)) of y, which is continuous 
in (x, $), such that when ¢ and (¢) are played the resulting velocity 
vector does not penetrate @. If more than one w-.is possible, we select 
some definite one. 

2. In the one-player game, G, which has the KE 


x = fx, $, oP) 


and playing space Æ, there is a semipermeable surface which we denote 
by 2. 

3. In a neighborhood of 2, the optimal control variables are locally 
continuous with y locally unique. Explicitly these properties mean: 
For any fixed pair x of and » a normal to 2 there, we can select the 

inimizi of 
ee H, v) = EnA, bv) 
so as to be continuous in x and » in some neighborhood of the above fixed 
pair. For a definite $, the corresponding maximizing y is a uniquely 
determined function of these variables and similarly continuous. 

4. A semipermeable surface, S, can be constructed in the standard 
way (Section 8.3) with 2 as initial curve. 

Then Z can be constructed so that its constituent optimal paths meet 
2 tangentially and the values of the optimal $ and y for S agree with 
those of Dat DO F. 

Proof. We choose coordinates so that @ is in the plane: x, = 0 with 
& on the side with z, > 0. Then for xe Zand all ¢ 


Aix $, WA) = 0. 


, Yn) be the normal vector in F& to 2. 


(8.5.3) 
Let ¥ = (0, ža... 


*It is always understood that we deal only with admissible ¢ and y, that is, all 
Constraints specified in the rules are always assumed satisfied. 
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By putting 
Olh, v) = EFF 4, v) 
the semipermeable condition applied to 2 is 
min O(h, WH) = 0 = OF, WE) (8.5.4) 


where $ = $(x) supplies the min and x € 2. 
The normal vector to 2 in &, to be used as initial value for S, is 
(Pis ¥a,--+5 Yn) for some »,. By putting 


OA%, y) = nfi, p, y) + O14, y) 
the semipermeable condition for F at 2 is 
min max Q(¢, y) = 0 = Q(4, 9). (8.5.5) 
From (8.5.5) 
AF, HA) < OG, 9) =0 
and from (8.5.4) Q4($, WA) > (4, HP) = 0 


If at least one of these inequalities were strict, we would have 


fix $, WA) = OF, HD) — AF, WA) < 0 


contradicting (8.5.3). 
Therefore, both of (8.5.6) are equalities. From the second, when we 


(8.5.6) 


construct 2, we can use ¢ in place of $, as hypothesis 3 justifies. From 
the first, and again from 3, the y(¢) on 2 agrees with p on F. The 
optimal strategies for F and 2 are equal at their juncture. Because this 
implies that the #, are also equal there, we have the stipulated tangency. 

Remarks. If more than one optimizing ¢ or y is possible at 2, we may be 
able to construct more than one semipermeable surface through 2. Only 
one, F, of this necessarily discrete set can have the continuous junction 
just exhibited. 

If more than one y is possible there will be an Z for each. The proper 
choice of # is dictated by the exigencies of the particular problem as will 
be seen in the examples of Chapter 9. 

The hypothesis 3 is necessary. We can utilize Exercise 8.3.1 as a 
counterexample (we adjoin —2 and 2 as bounds for y). It is clear that we 
may take as Vz. Calculation, with »3 = 1, yields for 2: 2, = 0, 
0 < z <1, z;=0. But the reader who has solved the exercise will 
know that on F, P = v/v; and that this ratio approaches 0 as we near 2. 
This discontinuity in the optimal y precludes tangency of the paths to 
2 and the conclusion of the theorem is untrue. 
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Observe that Y with its boundary 2 together constitute a semipermeable 
surface. 


COROLLARY 8.5.1. If 2 meets the BUP, the optimal paths of 2 meet it 
tangentially. 
Proof. In the coordinates of the last proof, if x € BUP then 


min max f(x, $, Y) =0 
$ y 


and let $ and # now supply the minimax here. By 3. we may suppose 
(after choosing a “branch,” if necessary) them unique near a juncture of 


2 and the BUP. At a point K of this juncture $ and # must thus be $ and 
ON 

Near K on the BUP we then have definite f(x) and (x). If they are used 
as control variables, then x must traverse a path in the BUP. The velocity 
vector (0, #2,..., %,,) of this path at K is the same as that of 2 there, as the 
control variables agree. Hence tangency. 


How are such ¥ to serve as the barrier of G? First 2 must meet the 
BUP. Then if x starts from S, under optimal play it first traverses its path 
in S, then one in 2 until it reaches the BUP. Here we may expect that x 
will escape into @ in the manner (the dotted curve) of Figure 8.5.1b. 
Thus until escape x has remained on a semipermeable surface and the 
outcome is neutral. 

By virtue of Corollary 8.5.1, 2 and the BUP have a common normal y 
when they meet. Thus the paths constituting must blend smoothly into 
those of the natural barrier. In such a case, then, the natural and envelope 
barriers together constitute one composite barrier. 

Observe that at a point of the juncture, the mutual optimal path, as one 
of S, is tangent to 2 and so by the corollary tangent to the BUP. Thus 
only through points of a natural barrier at which the barrier paths are 
tangent to the BUP is it possible to construct a 2. 

Due to this latter tangency it may well be that envelope barriers are the 
remedy to the disconcerting phenomenon depicted in Figure 8.5.3b. We 
know this is so in one case.’ 


Research Problem 8.5.2. In Figure 8.5.3b, let K be the point on the BUP 
which delineates the paths that dip under from those that do not. Is it 
generally possible to pass a 2 through K and construct an envelope 
barrier which takes over from the valid paths of the natural barrier? 


There is one last but important difficulty. When x is on 2 (which e ¥) 


° The isotropic rocket (Example 9.3). 
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it may be quite possible for P to cause penetration of @. In the language of 
the theorem’s proof, P has a determined optimal $r when x is traversing 


2. But this $ may not be the minimizing one for fi(x, $, y). Then 
penetration certainly can be forced. 
Further insight requires 


COROLLARY 8.5.2. Suppose ¥ actually separates the escape and capture 
zones of G. If there is a value ¢,"!, of ¢ such that, for x e 2 


il, $1, HA) < 0 


then there is a value y, of y such that, when 4,, y, are played, x will 
penetrate both @ and / into the escape zone. 

Proof. As 4, is not optimal for the semipermeable surface, the use of 
P(A) by E will cause x to penetrate 2 and such must be toward the escape 
side. The velocity x arising, being of G,, is tangent to &. As 2 is in the 
nonuseable part, E will certainly have a y, which leads to penetration of € 
into & when opposed by ¢,. By the Convexity Assumption (Section 2.7) 
any convex linear combination of »(¢,) and y, will be in E’s vectogram. 
If he chooses one with the lion’s share of the former and sufficiently small 
coefficient for the latter, the resultant y, will have the properties stated. 


Thus, should P attempt a capture with ¢, while x is on 2, E can retaliate 
by escaping with y,. But this seems to require decisions based on the 
opponent’s control variable, in violation of the definition of a strategy. 
Technically we can reply that definite $ and Ẹ exist at all points of the 
barrier; if the players navigate with them there is no hitch. 

Put practically, £ must have some foil for the above nonoptimal capture 
threat. Let him play in accordance a @, replacing @ as described in the 
“assuagement” near the end of Section 8.2. He responds, should P pull x 
beneath @,, but before, say, it reaches @,.. The simplest way seems to 
permit the above violation, our excuse being the short duration of the 
lapse. He meets $, with y, and, with a small enough e, gets into the 
escape zone in a very short time. 

Very likely this defect can be ameliorated, but the small interim of the 
unorthodoxy seems to occasion little practical difficulty In some cases at 
least, E could act as if $, were extreme (minimizes /;) and define a responsive 
strategy by defining ¥ as the y,, countering this extreme ¢ for x in a lamina 
below @,. 


X Possibly a vacuous statement, for f, may be independent of $. Then, of course, 
there is no possible penetration and no difficulty. 
44 Subscripts, in this proof, do not mean components. 
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8.6. SOME BRIEF EXAMPLES 


Example 8.6.1. Interception of a straight flying evader. Here P moves in 
the plane with simple motion and unit speed, while E is bound to a line 
(say, the z-axis), moves with speed w, and merely can select for his strategy 
one of the two possible directions of travel. Capture occurs when |PE] < 7. 

In view of P’s unquestioned ability to capture when w < 1, this case is 
trivial. Our interest is in the conditions which make possible the success of 
a slower pursuer (w > 1). 

The ideas here, although possibly not this very simple exemplification, 
pertain to a practically important question: When can an interceptor be 
successful against a faster attacking craft which travels a fixed straight 
course? The latter might be a ballistic missile with no provision for 
evasive maneuver or an attacking aircraft whose speed or plan permits 
little or no such. 

As we shall see shortly, this problem is very simple. We give the full 
formal solution to illustrate our context, 

Let coordinates be as in Figure 8.6.1a so that the KE are 


t = wy — sing 
ý = —cos¢ 


where —1 < y < 1 (preferable toy = +1). 





Figure 8.6.1 
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Figure 8.6.2 


The ME, is 
min max [—(», sin $ + va cos $) + wry] = 0 
$ v 


and so sin $ = »/p, cos $ = va/p with p = V7? + v and 


Y = g = sgn V3. 
Thus the ME, is 
—p + wo =0 
and the RPE 
f= -w +4, Y= %, = 0 
P 
j=”. 
P 
Capture occurs ((b) of the figure) when 
æ = l sins 
y = l coss. 


In the reduced space we have the semicircle of Figure 8.6.2 as @. We prefer 
to consider only y > 0. The useable part is defined by (note the vectograms 
sketched and that », the normal, = (sin s, cos s)) 


min max [—(sin ¢ sin s + cos ¢ cos s) + wy sin s] <0 
$ v 
or —1+ w |sin s| < 0 
; 1 
or Įsin s| < e A 
That is, if S is defined by sin S = 1/w,0 < S < 7/2, then the useable part of 


€ is l 
—S Ls <S (the heavy arc in the figure) 
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and so the BUP is given by 

s= oS. (recall o = sgn »,). 
Thus the initial conditions are 

x= olsin S, % = oø sin S 
y= lcosS % = cosS. 
The path integrals (of the RPE) are quickly obtained: 
« = o[(/ + 7) sin S — wr] 


y= (1+ 7)cosS. 
The slope here is 
cos S 
Oo —_—_— 
sin S — w 


which, because sin S = 1/w, is equal to 
sin S 
—0 
cos S 





and hence the barriers are tangent to & at the points where s = oS. We 
curtail them at their intersection (if w > 1). Thus we reach the result that 
the capture zone is the shaded region of Figure 8.6.2. 

The formal work could have been shortened. By observing that x and y 
do not appear on the right in the KE, we infer that Y; = 0 and then that 
the barriers must be straight. Hence, as soon as we know the useable part, 
we can draw the properly directed tangents from its endpoints at once. 

Note that if w = 1, the barriers do not meet. In this case, the capture 
zone is the strip 

lz]-< I 


The barriers are the lines z = +/ and are Static. 


Problem 8.6.1. Confirm the above results geometrically when w > 1. 
Problem 8.6.2. Discuss the “realistic” significance of the result when 
w=, 


To treat the interception problem mentioned earlier, we merely deprive 
E of his ability to change his direction of motion. Only one barrier now 
appears and the situation is shown in Figure 8.6.3. 


Example 8.6.2. Dresher’s one chance pursuit game. We reconsider 
Example 6.6.1, but we will generalize slightly by permitting motion with 
arbitrary speeds. Letting the vectograms be as Figure 8.6.4a, and x and y 
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y 
g 
Q 
x 
Figure 8.6.3 


being the components of the vector PE, the KE are 
t= wy + ¢ 
ġ=-—(w+1) wl. 
Capture as usual means 2? + y? < 1?. In the reduced space @ is para- 
meterized by s as in Figure 8.6.2 except now we admit the full circle 
instead of the upper half. The useable part is identified by 
min max [(wy + ¢) sin s — (w + 1) cos s] 
$ 
z = o(w — 1) sin s — (w + 1)coss < 0 


w >i w<l 
(6) 
Figure 8.6.4 
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where o = sgn sin s. Thus the critical angle S, connoting the BUP 
satisfies 

w+ 

w—1 





tan S = 


We can employ the short-cut of the previous example: the right sides of 
the KE, being free of x and y, lead to straight barriers. They appear as in 


(b) of Figure 8.6.4. 
In the case treated in Example 6.6.1, w = 1 and the barriers then appear 
vertical and parallel. 


Example 8.6.3. A synthetic problem. Using the KE of Exercise 8.3.1, take 
& as x3 > 0 and Ẹ as x = 0. The useable part is decided by 


min #3 = min (—%, + 1 — $) = —2, <0 
$ ¢ 


or x, > 0. The BUP is thus the 2, the x,-axis of Exercise 8.3.1, and we 
refer to the solution obtained there: 


zı = 4s7 — 47° 
Lo = S — 2r? (8.6.1) 
Lg = 287? — Tt. 
For s < 0, observe that z < 0 for small positive 7 and thus these paths 
cannot form part of the barrier. We thus suppose that s > 0. 
When 7 = V2s, the path pertaining to s returns to 7, = 0 and it meets 
this plane on the curve 
a, = —4/2s! 
ta = —3s, O<s< mo. 


Given any 23 > 0 and 2, we shall show that there exists uniquely s, 7 
with s> 0,0 <7< /2s such that the last two of (8.6.1) are satisfied. 
Thus the barrier meets every parallel in @ to the ~,-axis exactly once and 
so divides & into two parts. 

The proof involves merely some routine elementary algebra. Solving 
the last two of (8.6.1) for s and 7 yields for the latter 
rE =a, EVE + 32g 


3 
By choosing + = + we get just one possible positive r. Then 
S=%+277>0. 
From 
z = (2s — 7*)r? 
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follows that _ 
T< Ni 2s. 


Example 8.6.4. The dolichobrachistochrone. We have seen in Example 5.2 
that for starting points low in the plane—when y < w*—E can prevent 
termination despite all efforts of P. Thus the line y = w? appears to be a 
barrier. Like a natural barrier it is a semi- 
permeable surface through the BUP, but it is 
not tangent to @. This exceptional behavior 
is caused by the barrier’s being static. 


Exercise 8.6.1. Prove that line y = w? is 
actually semipermeable, static, and properly 
oriented. 


Problem 8.6.3. Show that above this line 
(y > w?) no semipermeable surface exists. 
Show that below it two pass through each 
point. This can be easily done geometrically 
as suggested by Figure 8.6.5. Finally, explain 
why the juncture of the transition surface and BUP confounds the tangency 
of @ and Z. 





Figure 8.6.5 


8.7. POSSIBLE OTHER SPECIES OF BARRIERS 


Our classification of barriers arose from an exploration of methodology 
and does not attempt exhaustion. That still further varieties may arise is 
indicated by 


Example 8.7.1. A further type of barrier. In this planar one-player game 
the KE are 
$ = w cos p 


ý = w sin y — u(x) 
where u(x) is a smooth function enjoying the properties: 
u(x) > 0 and u'(x) < 0 for all z. 
u(0) = w. 


It follows u(x) > w when æ < 0 and u(x) < w when z > 0. 
For @ we take the line through the origin 


L=S, y = as 


where « is a fixed number of either sign; £ lies above @. 
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We are going to state some results leaving the proofs to 
i y th , 
Let A be the point of © (see Figure 8.7.1a) such that pone 


Vu(A) — w|w = Jal 


so that s < 0. The useable part of @ lies to the left of A regardless of the 
sign of a. 
When z < 0 the family F of curves which are the integrals of 


dy = Vula} — w?jw 
dx 


are semipermeable. (They are sketched at (a) of the figure.) 

When « < 0, one member Z of F will be tangent to Ẹ at A. It con- 
stitutes a perfectly normal natural barrier (see (b) of the figure). 

But when « > 0, the barrier Æ is that member of F which passes 
through the origin O (see (c)). It is not tangent to and meets it at the 
nonuseable part. For starting points just above AO, such as the ¥ shown, 
capture does not occur immediately, but X follows a route as indicated to 
the useable part. 


Exercise 8.7.1. Attain these conclusions analytically. 


J: 





Figure 8.7.1 
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8.8. FUSION OF GAMES OF KIND AND DEGREE 


We have already discussed one aspect of this question in an early 
chapter. In a pursuit game, when the starting position is in the capture 
zone, P might adopt the attitude: “As I can capture, I shali do so as 
efficiently as possible and (say) minimize the time it takes.” If E corre- 
spondingly seeks to maximize, we have a composite game of kind and 
degree. f 

But there is a second possibility of fusing the two kinds of games. For 
x in the escape zone, P might say, “I cannot attain capture, but I will come 
as close to it as I can; I shall (say) minimize my closest distance? to E.” 

In both aspects there arises the question of the similarity between the 
optimal strategies (for both players) of degree and kind. The latter will be 
defined on the barrier; the former in a half-neighborhood of it at least. 
The two ¢ (x) [G(x)] thus defined can be regarded as one function on the 
closure of the half-neighborhood. If this function is continuous, the two 
types of strategies can be said to fuse continuously with one another. 

Whether or not there occurs this fusion is closely related to another 
matter, also of interest in its own right: 


THE ENVELOPE PRINCIPLE. If a pursuit game is solved with time of 
capture as the payoff, the envelope of the surfaces of constant V is the 


barrier. 


A beautiful instance of the principle’s holding is depicted in Figure 
5.5.4, where the envelope of the circular arcs of constant V is strikingly 
visible and does constitute a cross section of the barrier. In the homicidal 
chauffeur game, the principle holds for small V, but fails for larger, as we 
shall learn in Chapter 10. ee 

Lack of space necessitates postponement of publication of our researches 
into these questions. 

12 The mean distance is another possibility. 

18 He may be mentally adding, “Who knows? That fool of an E may not play 
optimally and I'll still get him.” 





CHAPTER 9 


Examples of Games of Kind 


The ideas of the preceding chapter are applied to a number of examples. 

The homicidal chauffeur game and its smoother counterpart, the isotropic 
rocket, are both solved in full. The game of two cars (Section 9.2) 
illustrates the complexities that may be submerged in apparently innocent 
problems. Its ideas here prove to be of practical importance in their 
similarity to those of collision avoidance. 

What we have called the lifeline and deadline games are a pair with the 
same kinematic background and apparently analogous content. But 
their solutions exhibit a remarkable distinction. 

The former game is a skeletal prototype of an evading craft trying to 
reach the (straight) boundary of large target area prior to capture by a 
faster intercepter. 

A naval interceptor (torpedo, ship, etc.) trying to nail a faster evader 
with a shoreline at its back is a broad instance of the deadline game. 

Variations of this game (Section 9.6) are rich in basic applications 
which, although simple in principle, are deceptively complex of solution. 
In the one-sided version the evader strives to pass between shoreline and 
pursuer. This leads to the cornered rat and corridor games. In the former, 
say, the evader is trapped in a “bay” and endeavors to slip past the pursuer 
hovering near its mouth. Still retaining naval terms, the latter game has 
the evader trying to slip past a pursuer in a channel or river (or it might be 
a football player passing a single tackler). We obtain the important 
critical channel width. 

This quantity also applies to the spacing of a patrol line of craft en- 
deavoring to frustrate passage of a faster evader. A similar idea governs a 
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circular patrol line seeking to prevent the escape of a centrally located and 
faster evader. 

In Section 9.7 we offer some suggestions as the applicability of our 
ideas to von Neumann’s general bounded pursuit games. 

We conclude by exhibiting but not exemplifying a method of solving 
dogfight (such as between aircraft with nonaimable weapons) games, 

There is also a strong possibility of applying these ideas to the theory of 
stability, but details, we feel, lie outside our compass. Such are sketched in 
the final section. 


914 THE HOMICIDAL CHAUFFEUR GAME 


We shall now investigate the conditions under which the more agile but 
slower E can avoid being run over by the faster but curvature-bound P, 

The variables, already discussed, are indicated in Figure 9.1.1. Again 
we encounter the KE 


á Wi è 
t=-—-— + wasin 

R yd 2 Y 
y= a$ — w, + w cos p, -l<¢<i 
which lead to the ME: 


min max (- A [yn — zv] — wW” + wa% sin p + ¥2 cos v) =0. 
$ » 


If we put 
A = yn — in 





š Figure 9.1.1 


1 In much of this chapter, sections and examples will coincide. 
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then $= sgnA=o 
and if p= v n? + vè 
then cos y= 2, sin 9 =~ 
P P 


so that the ME, is 


Wy 


—o—A — ww: + wp = 0. 


The RPE are found to be 


w 
where c= ~o. 
R 


Turning now to the initial conditions, & is defined by 


x = lsins, y =lcoss 
and the outward normal y by 
yı = sins, Yo = COS S. 


Thus the useable part is decided by 
EE max [7,4 + y.y] = min max [sin s(- a (l cos s)$ + wasin v) 
kad 


+ cos (2 (lsin s)6 — w, + we cos v)] 
== max [—w, cos s + wa cos(y — s)] 
d 


= w — w cos s < 0. 
Defining S by 


Oo<gS< kA ; cos S = We 
2 wi 
the useable part is thus specified by 
Isi < sS 


and its boundary, the BUP, by s = +S. 
We note that on & 


A = (l cos s) sin s — (l sin s) cos s = 0 
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so that we look to A, which is (in all of & as well) 
A= y(cra) — e(— cr) + n(—ez +w — We 2) - n(o — Ww rs) 
P P 


= Wi (9.1.1) 
Thus, on Y, o = sgn v, = sgns. We will work with the right barrier, 
taking 

c=l. 
The left-hand side is, of course, fully symmetric. The initial conditions 
used for the integration of the RPE are thus 
z= ÍsinS 


y = lcos S = 1% 
1 
n = sin S 


v = COS S. 
Integrating the last two RPE gives 
y = sin (S + cr) 


Vg == COS (S -+ ct), c= = (9.1.2) 
so that the first two are 
mc —w, sin(S + c 
o g ? (9.1.3) 


y = —cx + w, —w: cos (S + cr) 
which have, as may be readily checked, the integrals 
x = (l — wet) sin (S + c7) + R(1 — cos cr) 
y = (I — wer) cos (S + cr) + Rsin cr. 


Let us define the circles #, and %#_ as being concentric with the 
minimal turning circles and having radii in the speed ratio to them. That 
is, 4 , has center at (+R, 0) and radius (w,/w,)R. 

The barrier is the involute of X, which is tangent to @ as shown in 
Figure 9.1.2a for (9.1.4) are the equations of just this curve.” 


We have, either directly or by integrating (9.1.1) 
A= R{cos S — cos (S + c7)] 


so that, on Z, A ceases to be positive when S + cr = 2r — S. Here, 
from (9.1.2), the normal to the curve is (—sin S, cos S). It is not hard to 
see that this normal must lie along the line OB, which is the lower tangent 


(9.1.4) 


2In Chapter 10 we shall rediscover this involute by purely geometric methods. 
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from the origin to X}. If we attempted to prolong Z further, we would 
have to change o to —1. The extension would have to be an arc of an 
involute of #_; everything jibes here, for clearly the normal OB is 
tangent to both X „and #_. But the new involute would have to unwind 
counterclockwise from %_ and there would result a doubling back of Z 
an absurdity. Therefore Æ terminates, and the full (right hand) barrier is 
x arc of the involute extending from C to B, which is overscored in the 
gure. 

There is, of course, a symmetric curve on the left. The two may or may 
not meet, depending on the parameters, as shown in (c) and (b) of Figure 
9.1.2. In the latter case ((b)), & is not divided into two parts by #, and 





N 


—_- 


Figure 9.1.2 
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y 





Figure 9.1.3 


consequently all of & is the capture zone. That is, P can always achieve 
capture. From starting points such as X, E can force P into employing the 
devious route shown around @; such is what happens under a swerve 
(Section 1.5). Thus Z, although it does not delineate capture and escape, 
does demarcate the starting points from which optimal play will entail a 
swerve. 

When the involutes meet as at (c), we erase the parts of them beyond 
their intersection. The shaded curvilinear triangle is the capture zone; all 
the exterior region is the escape. We interpret the situation heuristically. 
Suppose the parameters are very much in E’s favor, that is, P’s margin of 
speed is not great, / is small, and the turning radius large. The most natural 
way for E to avoid being hit in such circumstances 1s simply to sidestep 
whenever capture seems imminent. From a starting position such as Y, it 
matters not what E does immediately. We can think of him as, say, 
remaining stationary. An attempted hit by P will bring x downward from 

Y; Eis not obliged to act until x is near Ø, when he sidesteps. This would, 
most naturally, mean that x traverses an involute near and external to B 
as shown at (c). He has sidestepped; the imminent danger is past until 
P aligns himself for a second pass at E, whence the same type of thing recurs. 

The shaded capture zone consists of those positions where E is placed so 
closely in front of the advancing P that, despite the former’s kinematic 
advantages, he cannot sidestep. As it is confined to a bounded set of 
starting points, while the escape zone is a set of infinite extent, we feel 
justified stating that whether or not the involutes meet is essentially the 

iterion for capture or escape. o. 
j With the aid of Figure 9.13 (equate the doubly drawn lengths), it is 

an elementary matter to write an analytic criterion. If y = the speed 
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ratio = w,/w, < 1, then capture occurs if 


b> vi — y? + siny — i (9.1.4) 


and escape if this inequality is reversed. 

For capture regions of other, even asymmetric, shapes than circular, 
the same type of criterion applies: whether or not the contacting involutes 
of X, and #_ meet.3 


9.1A. DOGFIGHTING A HIGHLY MOBILE TARGET 


For a craft, such as a single seater airplane with fixed machine guns, let 
the lethal area covered by the guns be of a form such as is sketched in 
Figure 9.1.4. Their fixed orientation relative to the craft, together with 
some stricture on motion such as bounded curvature; causes their 
effectiveness to be greatly dependent on maneuver. 

We can take the aircraft as P in the foregoing game, with the lethal 
area as capture region, and E the target. We could 
learn, say, how fast the target must be in order to be 
able to remain out of gun range. 

This problem will usually be of greatest interest 
when E has a speed greater than P and such neces- 
sitates extending the foregoing analysis. 


9.2. THE GAME OF TWO CARS 


In this and the following section we take up two 
problems of the same type as, but more difficult than, 
the homicidal chauffeur game. The present one leads 
to an analysis so formidable that we must be content P 
with a solution complete in principle, but not fully Figure 9.1.4 
displayed. Of course, any particular case—an in- 
dividual set of parameter values—could be computed to completion. 
The two car game was the subject of Example 8.1.1. It is just like the 
homicidal chauffeur except the evader too suffers the constraint of bounded 
curvature. Let w, and w, be the speeds of P and E and R, and R, their 
respective minimal radii of curvature. The dimensionality of the reduced 
space is 3, and there are many ways of selecting coordinates. We have 
chosen x, y, 6 as clarified by Figure 9.2.1. These appear to be practical 
from the standpoint of handling the somewhat oppressive differential 


* For sufficiently large regions, of course, no barriers will exist. For example of the 
capture circle contains X, and X. Then all of £ is the capture zone. 
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O CARS 239 
the RPE being 
e o 2 
z = cy — w, sin 8, Vy = CY, 
OK we Y = —cxz + w, — w, cos 8, Va = Cr, 


o 
6@=c—~% Oo, 
2 
We can also calculate that 


o 
Vs = wa(v; cos Ô — v; sin 0). 








A= wr. (9.2.1) 
ple We now turn to the initial conditions. We parametrize @ by 
curvature x= Isings 
= 1 
y = l cos s, 
0 = S, 


Figure 9.2.1 


as shown in Figure 9.2.2. If r2 = x? + y? we have from the KE 


rf = xt + yy = x(w, sin 0) + y(—w, + w cos 0) 
equations as well as being similar enough to the homicidal chauffeur game which on @ becomes 
to employ generalizations. Figure 9.2.2 depicts &, the reduced space; it is 
exterior to the cylinder of radius /. Of course, 6 is toroidal; it extends from 
0 to 277, and the planes of these two values are to be thought of as coinciding. 


It is not hard to write the KE: 


If = l sin s(w sin s3) — I cos 51(W, — We COS S3). 
f et 
Putting w= J Wy? + we? — 2wWe COS Sg 


Ww th 
| : | e see that the boundary of the useable part (where ê = 0) is given by 
t= — Ret wasin 6 

1 


. 33S Wy = Wo cos S2 i 
sins, = + I, cosy = 4 HER 9.2.2) 
Ww a 


ý = “2d — w, + Ww c08 0 
Rı 


ô =-"p+®y, -1< y<1. 
R,° R 


Let us put 
A = ny — v + 03. 
The ME, is then 


min max |- $ M14 + walr, sin 0 + », cos 0) + = Vey — wal =0. 
$ Y Rı R: 


Thus 
$= = sgn A 
P = 0z = Sgn V3. 


The RPE are found by our usual methods. To abbreviate we put 





Figure 9.2.2 
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The + means that, on each section of @ where 6 = s, = constant, the 
BUP consists of a pair of diametrically opposite points. 

We now must make a distinction as to whether w, > w or the reverse. 
In the former case, we see from (9.2.2) that sin sı never changes sign. Thus 
the two BUP remain more or less on opposite sides of @ and the use- 
able part roughly spans the upper half of @. Such is crudely shown in 
Figure 9.2.2. On the other hand, if w, < we, the sign change in s, (when 
COS Sz = w,/w.) means that the useable spirals around @. , 


Exercise 9.2.1. By means of vector diagrams interpret these conclusions 
geometrically. 


To save space we shall treat only the case where w, > we in detail. 
To ascertain ¢ and ğ at the outset, we must look to A and », on the BUP. 
As here 


: wi — We COS S 
v» = sin s, = 4 + 
WwW 


v = cos s, = + “25 
Vg = 0 
we see readily that A = », = 0 on the BUP. Thus the criteria are the 
signs of A and », 
Hence from (9.2.1) 
wi — wa COS s) 


> (9.2.3) 


= sgn A = sgn 7, = sgn (+ 


= (assuming w; > w2) + 1. 
Thus o, = +1 throughout the right-hand BUP. By symmetry we may 
work with this alone and from now on take c, = 1 and + = +. 


From the RPE 


oa = sgn ¥, = sgn (7; cos 0 — » sin 8) (9.2.4) 
= sgn [w, cos sà — wal. 


This changes sign, again supposing w, > Wa. Let S be the angle in the 
first quadrant such that 


wW 
cos S = <. 
Wy 


Therefore, in the intervals of s, below, o, has the values: 
0 to S, o= l1 
S to 27 — S, 0, = —1 
2r — S to 2r, 0 =l. 





[9.2] THE GAME OF TWO CARS 241 


Let us glance at the RPE. Clearly £ and ¥ do not depend on o» but 6 
does and so changes abruptly when s, = S and 27 — S. As s, increases 
through S, @suffers a sudden increase. This means that the paths from 
Sa = S— and s = S+ diverge from one another. We have a void, and it 
must be filled by a universal curve and its tributary paths. 

At 27r — S, on the other hand, 6 suddenly decreases; the surfaces 
formed by the paths from either side of 27 — S will intersect. They must 
be curtailed where they do so and here will be a dispersal curve. Note that 
as $ = 1 throughout this right-hand barrier, no instantaneous mixed strat- 
egy is necessary. 

Let us turn our attention to the universal curve. As it is a y-UC, in the 
notation of Chapter 7, we write the «, and f; from the KE and compute 
the y;: 

a; B; Yi 





i 
1 0 —cy+w.sind  (—w cos 0)wa/ Ra 
2 0 cæ—w +w cos (wasin 0)wa/Ra 
3 wR, —c 0 


Suppressing the factors w./R, and w,2/R,, the determinant is 
c(—y sin 0 + x cos 0) — w; cos 0 + w, = 0. (9.2.5) 
Which curve on this surface? Differentiate (9.2.5) with respect to r, 


and use the RPE (with replacing o,) to find the navigable paths which 
should comprise the surface. 


0 = c[—ysin 0 + 2 cos 0] + [c(—y cos 6 — xsin 0) + w, sin 616 
= c[c(y cos 0 + x sin 6) — w, sin 6] + [as above](c — = i) 
2 


= [c(y cos 0 + x sin 6) — w; sin is ğ. 
2 


Thus either y = 0 or the [ ]=0. The latter implies, together with 


(9.2.5), ct — W, + wacos 0 = 0 


cy — w, sin 6 = 0. 
These two equations belong to a curve in & on which, as we see from the 
RPE, z = ¥=0. Such a static configuration hardly suits our end. 
Therefore we take ẹ = 0. Turning again to the RPE we have, after 
replacing o, by 1 and o, by 0, 
£ = cy — wasin 0 


° 
y = —cxz + wy — w cos 0 


Do 


= 0, 
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From the third equation 6 = S + cr. Making this replacement in the 
first two, we see that they agree exactly with (9.1.3), the barrier equations 
of the homicidal chauffeur problem. Thus the projection of our universal 
curve on the x, y-plane is the involute barrier of that problem. 

This conclusion is quite harmonious with our general conception of a 
universal surface. It means that E, when playing the optimal neutral 
strategy, executes a sharp left or right turn until his orientation is suitable 
for him to play just as in the homicidal chauffeur game. E 

We have not yet integrated the RPE. Doing so, with the initial conditions 
for x, and v, already found, we obtain the result below, which can be 
readily checked. We retain c, and o, so that these paths fit all aspects of 
our problem. 


x = l sin (sı + cr) + R,o,(1 — cos cr) + R,o,(cos (s + cr) — cos 6) 


y = l cos (sı + cr) + Rio, sin cr — R,o,(sin (sg + cr) — sin 0) re 


6=s,+ (<- wt o) 
vı = sin (s; + cr) 
Ya = cos (sı + c7) 
% = Ro, [cos (S1 — S) — cos (s — s: + k os)]. 
In the above it is understood that s, is to be replaced by its value (9.2.2). 
We adjoin to the above 
A = R,o,(cos sı — cos (s, + cr)). 


We are interested in the smallest positive 7, called 74 and 7g, which 
annul A and v}. Without bothering with details we presume that the 
smaller of such 7 marks the termination of the barrier. We have 


p= Gas) Wee at 
Wy 
eho ir et 
Wy 


Ta a 2m —(sı — s) if o= +1 
w 


2 
R: 


e 2(sı aa S2) if O = —Í. 
2 


[9.2] THE GAME OF TWO CARS 243 


Figure 9.2.3 is a very crude attempt to indicate in part the appearance 
of the right barrier. The left one will be similar; note that on it, the 
dispersal curve (DC) emerges from S = 27 — S and the universal (UC) 
from s = S, and that the former curve moves toward rather than away 
from the plane of 6 = 0. 

The escape criterion is the meeting of these two surfaces and their 
bounding, together with the useable part of Y, a portion of @. But how 
can we tell if such happens? It seems a formidable computation indeed. 

Even the calculation of the dispersal curve is frustrating. The principle 
nevertheless is clear. In the three equations (9.2.6) we put c, = 1 (if we are 
working with the right barrier) and replace s, by its value from (9.2.2) with 
+ = +. We write the equations twice, with o, = + and —1 and consider 
7 and są distinct in each of the two sets. Then we equate the two x, y, and 
0. There are then three equations in the four unknowns consisting of the 
two + and the two s} They should have a one-parameter family of 
solutions which define the dispersal curve we seek. Of course, the 7 
obtained must be between 0 and min {74, 7} and the two s, must be larger 
and smaller than 27 — S. 

For any particular set of w,, Wa, Ry, Ra, /* we could arrive at the empirical 
answer. One way would be to plot cross sections of constant of the two 
barriers and see whether or not they all meet. The necessary equations are 
obtained easily from a minor manipulation of (9.2.6). 





Figure 9.2.3 


‘ As ratios only of speeds and distances matter, the effective number of parameters 
y pee P 


is at most three (for example, w,/w,, R,/l, Rail). 
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What changes if we treat the case of w, < wa? From (9.2.2) sı and 
hence o, changes sign when cos S = w,/wa, but from (9.2.4) o now 
remains constant on each barrier. It seems that the barriers will be broken 
by roughly the same kind of dispersal and universal curves as before, but 
now the latter will be a ¢-UC instead of a y-UC. We might conclude that 
it is always the player with the slower speed whose strategy is beset with 
discontinuities of singular surfaces, regardless of the curvature radii. 


Research Problem 9.2.1. Investigate the case of w, < wa more fully. In 
particular, what is the UC? Is it the same as the barrier in a pursuit game 
in which P has simple motion and E bounded curvature? 


9.3. THE ISOTROPIC ROCKET 


We return to this problem, solved with time of capture as payoff in 
Section 5.5, and construct the barrier. This time a precise escape criterion 
can be found, although sharp proofs for a few details appear to be re- 
calcitrant. We adopt the reduced coordinates v, y, v as before (see Figure 
9.3.1a), although we replace the X and Y by small letters. We also drop the 
friction force. Even though unbounded speeds for P are thus admitted, the 
problem gains in simplicity of formal mathematics without any sacrifice of 
principle. We will indicate later what changes result from a restoration of 
friction. 

The KE, readily derived, are 


¿= —F4sing + wsin y 
v 


y = F ĒĪsin ġ + wcos y — v 
v 
ù = F cos ¢. 

The reduced space (x, y, v with <? + y? > I, v > 0) @ is shown in (b) 
of the figure. As usual, Zis the radius of the capture circle. We parametrize 
@ by x = lsin s}, Y = l COS S4, v = Sa. 

Putting r = Va? + y? we compute the useable part by 

ri = at + yy 
= o(—F 4 sin ¢ + wsin v) + u(r sin $ + weos py — o) 
v v 
= w(z sin y — y cos y) — vy. 
Thus, on @, for the boundary of the useable part 


max lf = I(w — s, cos sı) = 0. (9.3.1) 
Y 
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(a) (b) 
Figure 9.3.1 


Looking along the z-axis we will thus see the BUP as the cylinder cut by 
the hyperbola 
vy = wl. 
(See (b) of the figure.) 
Using s = s as parameter we can write for the BUP, using (9.3.1), 


s= +V = (w/s)? 


y=" 
(9.3.2) 


v= s, S2> Ww. 


The rest of the initial conditions are, as is easily inferred from 1’s being 
normal to (9.3.2) and satisfying the ME, below: 


m= V1 —(w/s)? 


The ME, is 


; —U . 
max mo [r (= sin ġ + ¥5 cos 4) + wọ», sin y + v, cos y) — on] =0 
p 
where U = ny — vyz. 
If we also put 


A=VNV U?/v? + va, 


P= Vr? +n? 
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we have 
sin ġ = —, cosd?=—-8 
vPI Pı 
sing = Ë = 
(7 ; cos y = 
g 2 P2 
and the ME, is 
—Fp, + WPa — VV = 0. 
We are led to the RPE 
gap ll 71 i, = r2 
v pı Pe v Pı 
paar np, = an 
v Pı P2 v Pı 
2 
b= FË, PE EE 
Pı v Pr 


The closed integration of this system, with the foregoing initial conditions, 
presents its elementary difficulties, but the result is (we use as a convenient 
abbreviation W = Fr — w): 


2 2 2 — w? 
z= eal — WT + 1r], Vy = vs X 
v 2 v 
ys Hq; Fe iw + ie wr], asat pa 
vL \2 v 


v= vV — wW + We, n= —. 

The + distinguishes the right (x > 0) and left sides of £. By symmetry, 
we may restrict ourselves to + = +, that is, the right barrier. 

We also compute from the above 


r? = att y? = P H (st — wt IF + Hia (9.3.4) 


As we should expect r(0) = J, P (0) = 0. But we also observe that when 


s— wi — FI<0 
or when 

s<S=Vw+ Fl (9.3.5) 
then 7 (0) < 0. 
Thus when (9.3.5) obtains, our paths enter @ for small 7 and so are useless 
for constituents of a barrier.’ Let us call the points of the BUP which 


5 An instance of the phenomenon mentioned in Section 8.5 (see Figure 8.5.3). 
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mark the beginning of this phenomenon B+ and B-. They have the 
coordinates 


Vs? —w VFI 
Bt: x= +l 1 — 
S = S 

=]1ť 

4 S 

= §. 


Our next result is: at B+, the path is tangent to the BUP. 
For the tangent direction to the BUP is obtained by differentiating 
(9.3.2) and putting s = S. 








w w 
v, =l » Y l—, n=1 
= F ® So” 
Alo at B o — y? o Flw? o w 
f= -VFI G, y = s3 3 ar 


and these two triples are one a multiple of the other. 
The escape condition is that the two parts of the barrier meet or that 
there exist Tọ = 79(s) > O such that z(7)) = 0. From (9.3.3) this means 


that 
Fr? — 2wr + 21 


have a positive root. Thus the discriminant w? — 2F/ must be positive. This 
condition is sufficient, for we can take for 7) the lower root: 


_ wo vw — 2Fl 


= (9.3.6) 


To 


which is plainly positive. 
The above suggests that 


the escape condition is w? > 2F/ (9.3.7) 


but substantiation is needed. 

When (9.3.7) holds, let us call the curve where the barriers meet the 
crest. Its form is interesting. If we substitute the value (9.3.6) of ro for 7 
into the y and v of (9.3.3) we find, after a little manipulation that 


Y = VT. (9.3.8) 


Thus the crest is a straight line through the origin, in the (x = 0)-plane, 
with a slope (y/v) given by (9.3.6). 

When (9.3.7) holds, our work thus far leads to a barrier having the 
appearance of Figure 9.3.2. 
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The barrier, as thus far constructed, has the form of a semi-infinite 
tapering tent. But the right (on the figure) end is open, for beyond B, and 
B_ our barrier construction fails. What sort of semipermeable surface will 
seal it? This question proved the most difficult one thus far met in the 
subject. The answer will be given shortly. 

The remarks about sidestepping, in the discussion of the homicidal 
chauffeur game, apply here. There is no need to reiterate the argument 
that a meeting of the barriers implies possible escape for E. 





Figure 9.3.2 


If friction were restored, we recall that our function Q(r) (see Section 
5.5) is the radius of the cylinder of which the surface of constant 7 is part. 
The envelope principle (Section 8.8) holds; the barriers are the envelopes of 
these cylinders. Thus the meeting of the barriers is equivalent to the 
radius somewhere being 0. Therefore we can generalize the condition 


(9.3.7) by replacing it by min Q(7) = 0 


Exercise 9.3.1. Show that, with friction, the critical condition which 
demarcates capture and escape is 


(=) E — exp (- t=) ) = 1. (9.3.9) 


Replacing = by < signifies capture. 
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Show also that as k — 0, this condition reduces to the present case: 
w? = 2F], 


Research Problem 9.3.1, We have said nothing about the termination of the 
barriers. When the escape condition (9.3.7) does not obtain, we should 
expect the barriers to demarcate the necessity of something analogous to 
the swerve maneuver; indeed Figure 5.5.5 bears this out. Thus it is 
reasonable to expect that the barriers do come to an end. 





Figure 9.3.4 


Using the barrier equations (9.3.3) and forming the 3 x 2 matrix 
a(x, y, v) 
a(s, t) 
be can investigate the possibility of its rank falling below 2. Such happens 
when 


_ w+ V2s* -w — 2Fl 
F . 
Observe that for s > S, the argument of the radical is positive and that 


(9.3.10) 


THT 


Ww 
We ie 


so that the presence of 7p cannot interfere with the formation of the crest. 


Pe 
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Does the barrier actually terminate when 7 = Tp? If so, in kinematic 
terms, what is the reason? 
Problem 9.3.1. The Monotropic Rocket. Investigate the one-player game 
which differs from the present one only in that the thrust vector (of length 
F) always points directly forward. Thus P must move in a straight line 
and he has no volition. 

The BUP will be the same as before, but now the barrier paths will fail 
when s < S,, where S, is a root of 


s — wes — wF. 


But nevertheless the natural barrier is complete. (See Figure 9.3.4.) 
Obtain these results as part of the full solution. What is the kinematic 
significance of the paths that begin in the nonuseable part? 


9.4. THE ISOTROPIC ROCKET: THE ENVELOPE 
BARRIER 


To complete the solution of this game of kind we must apply the ideas of 
Part III of Section 8.5. We shall construct curves, 2+ and 2- in the non- 
useable part of @. Passing a semipermeable surface through them, each 
curve will be the envelope of constituent optimal paths. 

In this section, as most of results will be on @, we prefer new labels for 
its parametrization. We shall write 


x= lsin 
y = lcos0 


and the third coordinate shall be simply v. Thus we have replaced s, by @ 
and s by v. We shall use v, 0 as coordinates on @. 
As we did in Section 9.3, we can write on € 


= o(—FYsin g + w sin v) + uF sin $—v+ w cos p) 
v v 
= l[w cos (y — 0) — v cos 0]. 
Thus whenever 
peost ui (9.4.1) 
w 


we can pick y = Ẹ = y(v, 0) = 0 + cos™ (v cos 0/w) (9.4.2) 


so that F is 0 regardless of ¢. Hence, an essential hypothesis of Theorem 
8.5.1 is fulfilled. 
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Figure 9.4.1 


The condition (9.4.2) does not define ~ uniquely. There are two 
possibilities when (v cos 0)/w < 1. These are portrayed in Figure 9.4.1. 
The condition that the distance PE remain fixed as / is that the v and w 
vectors have the same projection (marked by braces) on the line PE. The 
choice between the dashed vectors is at E’s disposal. Because evasion is 
his objective, we presume that he selects the one on the opposite side of 
the line PE from the v vector. A brief study of Figure 9.3.1a shows that 
this choice renders sin (y — 6) > 0. 

Assuming y = #, we write, in the language of Theorem 8.5.1 the KE of 
the game G,. First we have, from the KE of G, 


4 = Lcos 66 = —F cos 0 sin $ + w sin ọ 
v 


LLRA E EEE T 
v 


y = —l sin 06 = F 
Multiplying them by cos 0 and —sin 6 and then adding leads to 
16 = ange v sin 6 + w sin (Ẹ — 6). 
v 


We can now write the KE of G;: 


; ae Ana? 
es 2 sich foe v» a (9.4.3) 
, ù = F cos ¢. 
a Z = vsin 0 + Vw? — v? cos? 6 
the retrograde form is 
jas sin $ — Z 
a i (9.4.4) 
v = —F cos ¢ 
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Figure 9.4.2 


In the (6, v)-plane, the BUP has the equation 


and B+ has the coordinates 


ve S=Vw+ Kk 
6=8 


where (recalling (9.3.5)) K = VFI and cos $ = w/S, sin $ = KJS. 
Our immediate objective is to draw a 


semipermeable surface of the game G, 
(with KE (9.4.3)) through Bt, such as 
the dashed curve in Figure 9.4.2. It is 
¢ to reach the v-axis at the v = v, > 0. 
The equations (9.4.4) are of the form 
a 


° 
6 =asind —c 
o 


v = —b cos $ 
with a, b, c>0. The vectogram, if 
b 


a < c, appears as in Figure 9.4.3. For 
semipermeability, if we are working on 
the right side (0 > 0), the heavy arrow 
must be chosen. If a > c, no surface locally exists. 

The heavy arrow is the one that minimizes 6/0 and so is determined 


Figure 9.4.3 
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by 
eee ee bom —b cos <a cos $) — (asin $ — c)(b sin $) 
dg dg 
= —ab + besin $ 
so that 
sin ¢ (or $ in the language of the theorem) = = 
and so 
PETE a E E A E (9.4.5) 
More conveniently, í 
AE O EE A E (9.4.6) 


dd 9 Jê- VZP- F? Q 
where Q = VZ? — Kij. 
At Bt we note that 


Z= S sin $ + Vw? — S®cos?B = K 
and 


Q = VK? — KYS = £ Jor +K)—Ki= rx, (9.4.7) 


The immediate problem can be stated in terms of passing an integral of 
(9.4.6) through B+. We note that at B+, this curve will be tangent to the 
BUP. For the former has slope 


and the latter 


d ( Ww } woes, BE SK 
~iz] = — sinf = wl -) ==. 
d6\cos 0 b=p cos? 8 w S w 
To finish the barrier problem completely, we should prove three things: 


(1) As we integrate the differential equation always we have a < c. 

(2) The integral through B+ reaches the v-axis at v, > 0. 

(3) The paths in the original & with initial conditions along the integral 
curve constitute a new portion of the barrier which joins to the old 
without leaks, and together the two seal off a portion of &. 
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We have not succeeded in proving these statements fully but what is 
unproven seems likely. We note in regard to them: 


1. The condition a < c is tantamount to Q? > 0. Now (9.4.7) shows 
Q is real at B+ and therefore the integral curve can be extended 
for some positive distance from Bt at least. 

2. We have proved (omitted here) that if Q remains sufficiently large 
along the curve, then (2) will hold. 

3. See Corollary 8.5.1 and the succeeding text. 


Finally, we show that, if the escape condition w? > 2F/ does not hold, 
then (1) and (2) cannot be true. For, when 6 = 0, 


Z=vVw — v? 
K? 
and Q =w. 
v 
We are interested in the latter for 0 < v < w. Near both ends of this 


range Q < 0. Its max occurs when 


4 
—<=-=-1+—=0 orwhen v=K 
v 


and here 
Q? = w? — K — K? = w? — 2Fi. 


Thus when the escape condition fails, Q? cannot be positive anywhere on 
the segment from (0, 0) to (0, w) and the integral curve from B+ must 
terminate before reaching there. 
But, with capture prevailing, a terminating, nonbounding barrier is just 
what we should expect! 
What does all this mean in terms of the kinematics of the original 
problem? A crude view of things is shown in 
Figure 9.4.4. First, when the outcome is 
neutral, P pursues E. The motion of each is 
elementary; E travels straight and P with a 
constant direction acceleration. Ultimately E 


E teaches the capture circle (X,); the motions 

of both have been such that the (relative) 

Xe Xi contact is tangential. Now £, by playing 
Figure 9.4.4 y = y (or by picking the direction shown in 


Figure 9.4.1) stays on the circle. During 
this phase P picks a ¢ yielding the heavy arrow in Figure 9.4.3. Now P 
and E in the realistic space are traversing paths of a fairly intricate type. 
Finally, when Æ arrives at some point X, on the capture circle he will be 
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able to leave it with no danger of immediate capture. Thus he has com- 
pleted a maneuver which might be called “sidestepping with contact.” 


Exercise 9.4.1, Complete Figure 9.3.2 by drawing in the new envelope 
barrier as best you conceive it. 


9.5. TWO REMARKABLY DISSIMILAR GAMES 
IN THE SAME SETTING 


The two points P and E have each simple motion in a half-plane 
bounded by a line Z. Their speeds are to be arbitrary; as only the ratio 
matters, we shall let P have speed unity and E, speed w(w 2 1). Capture, 
as usual, is to occur when the distance PE < 1. 


1. In the first, called the lifeline game, E’s objective is to reach # prior 
to capture and, naturally, P’s objective is the contrary.® 

2. The distinction from the former game is that now reaching will be 
fatal to E. That is, capture will be considered to have occurred if either 
|PE| < l or if E crosses Z. We perceive a generalization of both the 
wall pursuit game (Example 6.4.1) and the interception of a straight flying 
evader (Example 8.6.1). The novelty here is that Z is no longer confined to 
L but is free to roam one of the half-planes bounded by it. The name 
here will be the deadline game. 


Despite the parallelism of their formulations, the solutions are quite 
diverse. The former has an artificial barrier; the latter embodies one of 
envelope type. 

Clearly in the game 1, all is trivial if w > 1. For E is free to put as 
great a distance as he pleases between the two players and then to streak 
unhindered to #: all of & is the escape zone. Thus, here we will always 
suppose w < 1. 

On the other hand, game 2 is trivial if w < 1. For then E can always 
be caught, even if Z be disregarded. The latter’s presence can only make 
matters worse for E: all of & is the capture zone. We thus will suppose 
wp. 

For both problems we choose the coordinates shown in Figure 9.5.1a. 


The common KE are ; 
jı = cos $ 

Yo = W cos p 

ż = wsiny — sind. 


ê This contest would be same as the simple blocking game (Example 6.8.2) if we 
took w = 1. Because here we are interested in the game of kind aspect only, and in our 
former treatment the payoff was continuous (the distance E can advance toward 2 
before capture), the sidelines lose a great deal of their interest. Hence, due to the 
arbitrary speed ratio, we are essentially generalizing our earlier problem. 
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(b) 
Figure 9.5.1 


It is a familiar matter now for us to derive the ME,: 
max min [(», cos ¢ — vg sin $) + w(r, cos y + vs sin y)] = 0 
v $ 


so that, if 
A= Vn? +n, Pa = Vr? +r? 
then 
csé=—-, sinġ=”2 
1 Pı 
cos # = 2, sin p ==. 
Pe P2 
Thus the ME, is 
—pı + Wp, = 0 
and the RPE 
y 
HA =2 
Pı 
Ye SW 2 , v; =0 
P2 


The reduced space & appears in (b) of the figure. It is a quarter space 
(yı > 0, Y2 > 0) deprived of the interior of the cylinder &. The latter 
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corresponds to the points where |PE| = /; its axis lies on the 45° line of 
the ¥,42-plane and it is such that its vertical and horizontal (but not axis- 
normal) sections are circles of radius 1. The plane y, = 0, of course, 
corresponds to E’s being on ¥ and will be denoted by Li; Land g meet 
at the semicircle %. 


Example 9.5.1. The lifeline game. Thinking of x in £, we envisage its 
meeting X, as victory for E and @ as victory for P. Thus, if both a 
capture and escape zone exist they must be separated by a semipermeable 
surface which separates X, and @. Hence it must pass through %. Here 
is a pristine example of an artificial barrier. 

Conversely, if we can pass such a surface through % which separates 
é& such that Z, and @ lie one each in the two components, it is clear that 
these components will be the escape and capture zones. 

We parametrize X by 


Yı, =I coss 
Y =0 
. mT mT 
x = lsins, —-<s<- 
Sa 


A normal v to it and on it must satisfy 


y,(—/ sin s) + »,(/ cos s) = 0 
so that we may take 
vı = COS S, n = sin s. 


We invoke the ME, to obtain v}. As p, = 1, we have 
P: = oS V2 + sin? s 
w 


or v = +V (1w) — sin? s 


and + must be + to enable the vector to point into £. 

As the state variables do not appear on the right in the KE, all », = 0, 
we can write the equations for the barrier at once, the general formula 
being x; = (initial value of x;) + (constant) x r. We have here 


Yı = (l + T) cos s 
Yo = wry 1 — w sin? s (9.5.1) 
x = (l + (1 — w?)r) sin s. 


To gain an idea of the nature of this ruled surface, let us extend the 
paths backward beyond @. Putting 7 = —/: 


¥, = 0 
Ya = —wh/1 — (w sin s} (9.5.2) 


x = wl(w sin s). 
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Figure 9.5.2 


If we define the angle u = u(s) by 
sinu = wsins 


so that u ranges from —f to +f, where £ is the first quadrant angle with 
sin f = w. Thus the curve (9.5.2) is the circular arc of radius wi(< /) in 


he zy,-plane, 
the Yo Yı = 0 
Yo = wl cos u 


—ß Ku <$ 


æ = wl sin u, 
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and the surface.(9.5.1) can be drawn by drawing straight lines between the 
corresponding points of it and 4. If w < 1, the result is as sketched in 
Figure 9.5.2, which is a sort of half horn through % and clearly fulfills its 
office of separating Z, and ¢. 

To get an idea of what’s what in the realistic space, let us imagine the 
curve which is a horizontal section of this surface at height y, >/. In 
the realistic space this situation corresponds to P’s being at a distance Vi 
from &. The section itself can be regarded as a picture of the realistic 
space: the capture circle will be the section cut of & and the section of Z 
will appear as the other curve in Figure 9.5.3a. If E starts from any point 
below this curve he can reach Y with impunity; from any point above 
P can catch him first. 

If w = 1, the barrier surface of Figure 9.5.2, coincides with the upper 
half of the cylinder @; the curve in Figure 9.5.3 coincides with the lower 
half of the capture circle (see (b)). What is the meaning ? 

The answer is a static barrier. 

Such requires that ¥, = y, = ż = 0. From the KE, this can occur only 
if cos $ = 0, sin $ = +1, w= 1, sin y = +1, cos y = 0. We can then 
normalize so that p, = p, = 1 and we must have further 


vy, = 0, v = 0, Vg = +1. 
The ME then being 


max min [—(+1) sin ġ + (+1) sin y] = 0 
v $ 
plainly it is satisfied by the above ġ and y. Then in the reduced space, 


static semipermeable surfaces are planes normal to the 2,-axis. For our 
problem we utilize the halves of the two such planes which are tangent to 


t 
| 
1 
<— 


| 
I 
Ez! cz | EZ 
| | 
l | 
CZ / EZ 
P Ne pa. 
w<l w=1 
(a) (b) 
Figure 9.5.3 
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@ and lie below. Such join into our former semicylinder to form a single 
smooth surface. 

Returning to the realistic space with fixed y,, we find we have adjoined 
to the barrier the two dashed half-lines of Figure 9.5.3b. The above + 
takes a different sign on each; it is easy to see that the arrows drawn 
correspond to the flight directions of both players. 

Note how this barrier agrees with our former analysis of the simple 
blocking game. 


Problem 9.5.1. Show that (if w < 1) the same Z can be reached by the 
geometric method. Consider the locus of points C such that 


w(|PC| — 1) = IEC]. 


Show that the barrier corresponds to the set of positions such that this 
locus lies above # and is tangent to it. Explain why. 


Research Problem 9.5.1. Solve the lifeline game when & is replaced by a 
circle. The playing space may exterior or interior to it. One might 
generalize further by taking X as an arbitrary curve. 


Example 9.5.2. The deadline game. In Figure 9.5.1b both Z, and @ are 
now anathema to E. Their union can be regarded as the useable part. 

Let us consider the semicircle #. It corresponds to positions where E is 
on £ and |PE| = l such as in Figure 9.5.4a. Clearly E can break away 
only if his velocity component along PE can be made to exceed 1, that is, if 
w sin s > 1. Defining £ by sin B = 1/w, cos B = Vw — 1/w this condition 
is also s > B. Now this s and that of Figure 9.5.1b are the same, and the 
only portion of 4 which might be considered as nonuseable is the arc of 
X called X, where —p <s < f. Thus the two points of # with 
s = +6 are all that we can salvage of a BUP. 

An envelope barrier seems a likely answer. To construct one we shall 
follow the scheme of Theorem 8.5.1. 

We will redefine ¢ and y as shown in Figure 9.5.4b. To keep the 
distance PE fixed at /, E should choose y so that the projection of both 
velocity vectors on PE are equal. Of the two ways of doing this we shall, on 
intuitive grounds, suppose that he strives to make s increase; for example, 
in the figure he will endeavor to move counterclockwise around the 
capture circle. Then clearly 





aA 
cos y = — Cos 


= 


(9.5.3) 


sin ğ = 1 — cos} 
w 
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Figure 9.5.4 


and for short we shall call the radical R. For coordinates on & we shall 
use s and yp, As x = lsin s we see from the figure 


(l cos s) = 4 = w sin (s + y) — sin (s + 4) 
sin s cos $ + R cos s — (sin s cos $ + cos s sin $) 
= (R — sin ġ) cos s 


and also 

Ja = —w cos (s + y) = R sin s — cos ġ cos s. 
Adjoining (9.5.4) 

Iš = R — sin $ 
we have the KE of G,. To obtain a semipermeable surface we proceed as 
with the isotropic rocket game: 

_d. d ; sin s cos ¢ sin ġ i 
0 = IS — ġa — ġa — lš = (R — sin (Hes co esing 4 coss sin ) 
dé Y2 Yo dd ( $) R $ 
. cos ¢ sin $ 
—(R sin s — cos s cos ġ$) Ce oe cos ġ 

which reduces to 


w? sin $ cos s + (w? — 1) cos ġ sins — Rcoss =0. (9.5.5) 
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After further reduction (9.5.5) becomes a biquadratic equation in cos 4, 4 
For each root there is still a sign choice in evaluating sin ¢. However, - 


there are but two of these four possibilities that are solutions of (9.5.5), 
Such correspond to the extreme vectors of the vectogram. From the 
nature of our problem we wish 6/y, negative, for during neutral play s 
clearly increases (to f) and ya decreases (to 0). Only one possibility 
achieves this. It turns out to be, and the reader may check it, 


w cos s 
cos ¢ = —— 
Q 
sin? = Tews (9.5.6) 
Q 
R= w(w — sin s) 
Q 


where 
Q=vV1 + w? — 2wsins 


and these values occasion, when substituted into (9.5.4), 


Ja = — eens). which < 0 when 6 < £. 
2 (9.5.7) 
w? — 1 
oO 
The quotient yields 





S= 


dys ( wl . 
— = —|——_](1 — wsins 
ds we — 1 ( ) 

an equation which is to be integrated with the initial conditions: s = £,” 
Ya = 0. We obtain 


n= aL 





i w(k — s — w cos s) (9.5.8) 


where k = $ + w cos B = B + Vw? — 1. There is no reason why s should 
not parametrize 2, the envelope curve. The other two equations are easy 
to obtain. As y, — Y = l cos s, 








n=- (wk — ws — cos s) 
w— 1 
(9.5.9) 
h= (wk — ws — w? cos s) 
we — 1 
æ = lsins 


7 We are working here on the side of £, where x > 0. 





‘9.51 TWO REMARKABLY DISSIMILAR GAMES 263 


which are the equations of 2, or rather the branch for which x > 0. 
Clearly the range of s in (9.5.9) is [B, 0]. 

To obtain the semipermeable surface is easy; it is the union of 2 and 
its half-tangents properly oriented. The first equation is obtained, for 
example, from (9.5.9), if y,(s) means the latter’s right side, by 


Yı = yi(s) — y1'(s)r. 


The minus sign appears because s decreases (from £ to 0) as we proceed 

retrogressively along 2. Of course, the 7 appearing will not measure time 

of progress; this would be so only if s did along 2 and it plainly does not. 

But the distortion of 7 does not change the surface, only its parametrization. 
Thus the semipermeable surface finally is 





[w(k — s) — cos s + (w — sin s)7] 


at 
nyi 


(9.5.10) 





Yo = [w(k — s) — w? cos s + w(1 — w sin s)r] 


w— 1 
x = l[sin s — ros s]. 


Because we are only working here on the x > 0 side, we break things 
off at the x = 0 and adjoin a symmetric image. Figure 9.5.5 shows the 


J 





x 


Figure 9.5.5 
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Figure 9.5.6 


g — e 


final envelope barrier. The arc CB is the envelope curve J, with s = f at 
Cand 0 at B. As 2 and & are tangent at C, as has been shown, the 
tangent line CA to 2 lies in the x,y,-plane. The remaining half-tangents 
to 2 are extended until they meet the x = 0 plane, which they do at the 
crest AB. This curve, as may be easily shown, rises (y, increases) on 


proceeding from A to B. 
Let the letters A, B, C also designate the values of y, at the corresponding 


points. Then from simple computations 


wl 


Vw — 1 


B= l [wk — 1] = 
we — 1 


A(=y, at A) = 











2 


Ak (sin + Vw =1) — 1| 


(9.5.11) 


we — 


Jw —1 
= 





C=IcosP=! 


To get an idea of what goes on in the realistic space, we consider, as 
before, sections of constant y,, which can be regarded as pictures with P 
fixed and E variable. When y, < C or >A capture can be forced by P for 
no position of E; the capture zones for the remaining case are depicted in 
Figure 9.5.6 as shaded regions. 


Exercise 9.5.1. What are the equations of the barrier sections in Figure 
9.5.6? 


Exercise 9.5.2. Investigate the limiting case as w—> 1. Show that when 
w = 1, the barriers are static. Their sections in Figure 9.5.6 would appear 
as inversions of those of the lifeline game: the configuration is the upper 
half of the capture circle plus half-tangents extending downward to 2. 


Research Problem 9.5.1. Investigate the paths of P and E in the realistic 
space corresponding to x being on 2. Is there any simple interpretation 
to the curved path of P? 
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Research Problem 9.5.2. Generalize by allowing Z to be the arc of a 
circle. 


9.6. EXTENSIONS AND APPLICATIONS OF THE 
DEADLINE GAME 


To start we take up (always w > 1) 


Example 9.6.1. The one-sided deadline game. We envisage E as starting 
from, say, far to the left of P who is a distance > / above Y. The object 
of E is to move to the right and pass between P and Y without being 
captured. Of course, P’s object is the contrary.® 

The distinction between this problem and that of the previous section is 
that now symmetry is abandoned. The same barrier suffices here, but we 
no longer curtail it when x = 0. That is, the equations (9.5.10) still supply 
the relevant semipermeable surface, but the parameter range is to be 
enlarged tos < 6,7 > 0. In the counterpart of Figure 9.5.6, we retain only 
the right boundary of the shaded region, producing it if necessary until it 
meets Z. 

But here we encounter a novelty. Our tenet that the barrier separate & 
is violated. However, validity has not absconded but changed its nature. 
Topology has intruded. Before expounding further, it is well to look at 
our now asymmetric barrier. 

Consider the curve labeled 2 in Figure 9.5.5. If it is produced beyond 
B it will, as its equations show, spiral around the cylinder @. The barrier 
then, the union of the half-tangents, will be a sort of flared helicoid. All 
these rays extend upward, for we see from (9.5.10) that dy,/dr > 0. All 
then will meet a horizontal section provided they emanate from a point of 
2 lying below its plane. Thus a high such section will consist of a spiral 
whose number of turns is greater with increased height. Some cases with 
progressively greater y, are shown in Figure 9.6.1. 

The situation fits a game whose payoff is a non-negative integer. 


Example 9.6.2. Looping the blockader. Let £ start, say, from the left, as 
in the last example, and finish on the far right. The payoff is the number 
of counterclockwise revolutions he can make about P without being 
captured or touching Y, or it is —1 if E cannot even pass without being 
captured. 

It is easy to apprehend the solution. The curves of Figure 9.6.1 (for a 
fixed position of P, of course) separate starting points of different Value in 
much the same topological way as the sheets of a Riemann surface 


è In terms of football, we can think of a ball carrier trying to pass safely between 
a sideline and a slower, opposing tackler posted near it. 
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Come au 


separate the branches of a function.” For example, if E starts from C, D, 
F, the Value is 0, 1, 2, respectively. 


Example 9.6.3. The cornered rat game. The rat E is entrapped in a corner 
(of any angle):by the cat P.° When is escape possible? 

In certain circumstances—one is shown in Figure 9.6.2a—by drawing 
the barrier sections of the type just discussed to each wall we can delineate 
the capture zone. Here E cannot escape if his starting point lies within the 
shaded region. iv 

Further analysis of this problem will be presumed similar to that of the 
one which follows and so will not be discussed separately. 


Example 9.6.4. Patrolling a channel. Let E be confined between two 
parallel lines of distance apart L. His objective is to get past P without 
being captured. 
Figure 9.6.2b is an instance in which E seeks to escape to the far right. 
Again two barrier sections mark off a capture zone (shaded) of starting 


Figure 9.6.1 





(a) (b) 
Figure 9.6.2 


? For example, note the Value changes from 0 to —1 in going from A to B via the 


dashed line in the figure. : ; 
1° Speeds, motions, and capture are as in the deadline game and the same i 


ensuing examples. 


s true of the 
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points from which £’s passage can be prevented through proper play by P. 

In both these problems it is intuitively clear that for sufficiently favorable 
values of the parameters E should be able to pass P without capture from 
all (or possibly all but some insignificant subset of) starting points. 

Let, us clarify a notion of intersecting barriers for sharper reasoning. 
Suppose, in the analysis of some general game of kind, we encounter two 
candidates for barriers which intersect as shown in Figure 9.6.3, where 
n = 2 for simplicity. Observe that they are oriented oppositely from our 
earlier cases such as the homicidal chauffeur game. There we saw that we 
were justified in erasing the barriers beyond their intersection; here we 
shall show that we should discard the entire barrier. 

We suppose the barriers not static, that is, under optimal play x traverses 
them toward @. Let x start from X,, which is in the capture zone but 
very close to one barrier #,. Both players’ adoption of the optimal (very 
near the neutral) strategy will occasion a motion nearly parallel to A. 
Ultimately x will reach X, near the intersection. If the same strategies 
persevere, x will cross Z, and E will escape. 

Note that P cannot prevent this escape. The intersection of the barriers 
does not entail constructive guessing by a player of which strategy—that 
relevant to Z, or @,—his opponent will use; an instantaneous mixed 
strategy is futile here. Let us suppose that at X,, E decides on the strategy 
pertaining to #,. If P uses this, too, we have seen that x crosses Z, and 
E escapes. If P does anything else, E’s strategy will occasion x’s crossing 
of Z, and again he escapes. Thus al/ the nearby region belongs to the 
escape zone. 

To see that this kind of intersection does occur in our game when L is 
sufficiently large, refer to Figure 9.6.4. To take into account the added 
side of the corridor we adjoin, in the obvious way, the new coordinates 





Figure 9.6.3 
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(d) 


e) 
f Figure 9.6.4 


L — y, and L — yı. The two 2 curves are intertwining spirals as shown at 
(a). At (b) we have a vivid attempt at depicting part of the intersecting 
barriers; the reader can convince himself that the orientation is as in 
Figure 9.6.3. Thus we conclude that whenever the channel is so wide that 
the two barriers intersect, as at (c), then E can pass from any starting 
position. We will show below existence (and find it) of a critical channel 
width where this kind of intersection begins, regardless of P’s position in 
the channel. pe 
Let L, denote the critical width at which intersection commences if P is 
centered in the corridor (y, = 4Z,). The sectional barriers must both 
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meet the capture circle with s = m/2 as shown at (e). Giving y, and s 
these values and 7 = 0 in (9.5.10), we ascertain that 


or (9.6.1) 





The assertion we are after is 


If L > L., E can pass from any starting position (P cannot guard the 
corridor) but such is not true if L < L, (9.6.2) 


We need but show that L > L, is the condition that the two barrier 
sections intersect. We have seen that this is true when P is centered in the 
corridor. To prove it for arbitrarily located P, as at (d), it suffices to show 
that, when L = L, the two barrier sections meet at a point of the capture 
circle wherever P is. Such is tantamount to showing that the two spirals of 
Figure 9.6.4a coincide when L = L,. 

The equation of one spiral is given by (9.5.9); those of the other by 
replacing y, by L —y,(i= 1, 2) and s by w—s. Our result follows by 
noting that when L = L, the final replacement follows 
automatically. For example, from (9.5.9) and (9.6.1), 





(C25; 





l 7 
L=- y= e z[»(r — z) — [wk — ws — cos sh} 





=<" {wk — wr — s) — cos (r — 9)}. Ges 


Thus the assertion (9.6.2) is proved. 





Example 9.6.5. The patrol line. Given a row of equi- CrP 
spaced pursuers, under what conditions can a single 
evader E pass through the row without being captured? 
By imagining corridor sides placed between each Figure 9.6.5 

pair of adjacent P as shown in Figure 9.6.5, it would 

appear as though, by the principle of reflection, this game reverts back 
to the previous one. Thus a sufficient condition that the patrol line be 
effective is that the spacing < L,,. (9.6.3) 





Research Problem 9.6.6. The Patrol Circle. Given a set of P equispaced 
around a circle, when can they effectively prevent the escape of a single E 
who is initially within the circle? 
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This problem bears the same relation to the cornered rat game as the 
above does to the corridor game. 


Problem 9.6.1. Treat the limiting cases of the above games when w = 1. 
Show that in the cornered rat and corridor games, a capture zone always 


exists. 


9.7. FURTHER GAMES 


The examples in this section have not been fully solved. But they are of 
interest and possibly of importance. Because no more than suggestions are 
given for their treatment, the reader can regard them as research problems. 


Bounded pursuit games 

Here P and E move in the plane, both with simple motion, with E having 
the greater (or possibly equal speed). But E (and possibly also P) is 
confined to a subregion @ bounded by one or more curves. This class of 
problem was suggested to us by the late J. von Neumann. 

The deadline, cornered rat, and corridor’ games are all instances. 
Can we handle other regions? 

Suppose Z is the interior of a circle. If it is of large radius, it is reason- 
able to suppose that results of the type depicted in Figure 9.5.6 will hold 
when P is near £, now dished upward. Do the shaded regions retain their 
validity as capture zones? Certainly when £’s starting point is within them 
he will be captured. But what of unshaded starting points? If it is true 
that the shaded region of the figure is the only capture zone and it 
changes continuously with P’s motion, then E will be able to escape 
perpetually. For if not there would occur an instant when E would be 
able to cross the boundary of the zone; but he cannot be forced to do so 
because the boundary is semipermeable. 

If Z is the interior of a polygon, we might similarly endeavor to apply the 
cornered rat game to positions where P is near a vertex or the deadline 
game when P is near the central part of a side.” 

We hesitate to stress these concepts overmuch, for it seems likely that 
there will be significant cases in which all of @ is the capture or escape zone 
and barriers will not be the appropriate tool. 


11 Not exactly, for here E is not interested in passing P and can always escape. But 
if the corridor is cut off by a cross boundary at one point (semi-infinite strip) or at two 
(long, narrow rectangle), then from some starting positions E must pass P in order to 
survive and our previous ideas are applicable. 

12 Especially the former. For if @ can be contained in a half-plane, then a non- 
vacuous capture zone will exist. For this zone of the deadline game will a fortiori hold 
for a more restricted region then the containing half-plane. 
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Connectivity must have a bearing (assuming P is similarly confined to 
&). For in some cases P can clearly never catch £ if he has to chase him 
around a closed circuit. 


Dogfight games 

Let P and E move in, say, the plane, each with such kinematics that 
their directions of motion are state variables (Example: bounded curva- 
ture). Let each have a capture region which extends forward (in the 
direction of motion) of his position. Figure 9.1.4 is a typical example. 
Each player strives to get his opponent within his own region prior to his 
being himself so caught. Thus we exemplify conflict between two single- 
seater airplanes, each armed with unturnable guns that point straight 
forward only. 

Of course, a partie may be a draw in that neither player may be able to 
force the other into his capture region. To simplify the discussion we shall 
suppose this possibility excluded. 

In the reduced space, @ will consist of two surfaces, @, and @,, which 
correspond to the boundaries of the two capture regions in the realistic 
space. As play proceeds x moves about in & until it first penetrates one of 
the @,. Which one determines the outcome. 

We see the technique of solution at once. The two @; will intersect, for 
both players being within their opponent’s capture region, either one only, 
or neither, must all be admissible positions in any reasonable simulation 
of reality. Let the intersection consist of one (or more) curves. We pass a 
properly oriented semipermeable surface through these curves. It should 
separate & into two parts which are the winning starting positions of the 
two players. 

Naturally dogfights may be formulated in other ways than a pure game 
of kind. For example, victory may require that one’s opponent remain in 
one’s capture region for some stipulated duration. Or a continuous payoff 
could be adopted which is an integral of the time of the vulnerability of one 
player less that of the other. 


Battles of extermination 

The players commence with certain numbers of “‘men’’® and the kine- 
matic equations describe the mode whereby these numbers are altered 
during play. The player loses the game who first loses all his “men.” 

Thus the contest is a game of kind and a barrier should delineate the 
starting positions from which one or the other player wins. We have seen 
how to construct such in discrete cases in Chapter 3. To adopt the same 
principle to continuous models, in the light of this chapter, is not hard. 


18 Soldiers or other munitions in a battle, checkers on a draughts board, etc. 
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9.8. APPLICATION TO STABILITY AND CONTROL 
Such problems are one-player games with our customary KE: 
&, = fx, $). P= lsh 
and subject to the supplementary condition 
SO, 0) = 0.4 (9.8.1) 


Here x is regarded as the “error” of some possibly unstable mechanical 
or electrical system and the ¢, are control variables whose function it is to 
obviate undesirable digressions from the central equilibrium state. This 
state is taken at x = 0, which is the interpretation of (9.8.1). The designer 
has the option of making ¢ a function of x—just our conception of a 
strategy. f 

various criteria for stability are extant, generally based on the asymptotic 
or ultimate behavior of the differential equations arising when ¢ = ¢(x) 
in the KE. 

It seems reasonable that the theory of games of kind can offer an 
alternative approach. The boundary to all possible deviations from 
equilibrium will be a barrier and our techniques should be able to ascertain 
it. Indeed the idea is used in the concept of “‘controllability.”" 


14 See, for example, La Salle’s survey article [7] and the example in the Appendix. 
16 See References [5]. 





CHAPTER 10 


Equivocal Surfaces and the Homicidal 
Chauffeur Game 


10.1. INTRODUCTION 


The remarkably eclectic and instructive game of the homicidal chauffeur 
has appeared fragmentarily in several earlier sections. To weld the bits 
into the full solution, we must introduce, in addition, the rich variety 
already embodied, a new type of singular surface. It has, of course, wider 
interest than for this one problem. 

It is termed an equivocal surface, for one of the players has the choice of 
two distinct optimal strategies at each of its points. The phenomenon 
cannot exist in a one-player game; there is no counterpart in the calculus 
of variations. We will expound its theory in Section 10.5 by a detailed 
discussion of an example typical enough to embody all salient features. 

For the homicidal chauffeur, both the games of kind and degree are 
interesting. We solve the former here by a purely geometric construction 
and treat the latter by methods partially so. This more transparent 
approach will clarify, confirm, and complete certain earlier analyses. 


10.2. THE HOMICIDAL CHAUFFEUR: GEOMETRIC 
SOLUTION OF THE GAME OF KIND 


We will exploit the symmetry of the game by speaking almost through- 
out as if E were on or to the right of P’s line of travel. In other words, we 
will work in the right half of the plane in which lies &, the reduced space. 

To solve the game of kind we must construct the barrier. We have done 
so by our standard technique in Section 9.1, but in the one immediately 
following we shall obtain the barrier geometrically. 
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The game of degree, with time of capture as the payoff, hinges strongly 
on the barrier. As we have seen and will see again, it consists of two arcs 
of involutes of the circles we have termed %, and .%_.1 When these arcs 
intersect, P cannot compel capture unless x is in the curvilinear triangle 
bounded by them and @ (see Figure 9.1.2). But such a position, as we 
noted before, is a setup: E is planted directly in front of the oncoming 
vehicle. Thus we are justified in regarding the intersecting barriers as the 
general criterion of escape. 

Of course, we can solve the game with capture time in this case (the 
solution exists only in the triangle), but it is of much the greater moment to 
work with nonmeeting barriers. These curves still play an important role, 
They delineate the positions in which optimal play leads to a straight- 
forward chase from those where a swerve? maneuver occurs. 

We recall that Æ travels in the plane with simple motion of speed wg. 
Thus his vectogram in the reduced space @, that is, in a plane rigidly 
attached to P’s vehicle, is evident: we still have a circular vectogram of 
radius wz. 

But P’s vectogram (the ¢-vectogram) is not so apparent. Our first 
step—Lemma 10.2.1 below—is to describe it in geometric terms. Recall 
that P travels at the fixed speed w, (> wą) and with his radius of curvature 
bounded absolutely by a given R. He navigates by choosing his curvature 
¢/R (with —1 < ¢ < 1) at each instant. 


LEMMA 10.2.1. At any point X of £, the ¢-vectogram is constructed by 
the steps (see Figure 10.2.1a): 

|. From X¥ draw a downward vector XA of length w4. 

2. Through A draw a line H perpendicular to OX (O = origin). H 
contains the headline of the vectogram. 

3. From X draw the vectors XU ,, terminating on H, which are respec- 
tively perpendicular to the lines from (O, +R) to X. These vectors are the 
extreme members of the vectogram, XU, corresponding to ¢ = +1. 

Proof. Put r = R/¢(—1 < ¢ < J), so that the center of the curvature 
selected by P is C = (r, 0) as in (b) of the figure. Put d=|CX|. 

Now P’s pivoting about C in the realistic space is equivalent to X’s 
pivoting about C in £ in the opposite direction but with the same angular 
speed. The resulting velocity of X will be XU perpendicular to CX and of 
magnitude w,d/r. 

Now consider the triangles OCX and AXU. They have two pair of 
mutually perpendicular sides, and the length ratios of these sides is w,/r in 
both cases. Thus they are similar, and the third sides OX and AU are also 


1 They have centers (+R, 0) and radius Rw,/Wy. 
2 See Section 1.5. 
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eas 


Figure 10.2.1 


perpendicular. This proves the contention in 2 and 3 follows because of 
the bounds on ¢. 


LEMMA 10.2.2. The semipermeable direction (properly oriented for the 
barrier) at X is constructed by first drawing a circle of center U, and 
radius ws. The sought direction is that of the lower’ tangent (XD in 
Figure 10.2.2a) from X to this circle. The local optimal strategies are: 
$ = 1 and @ is such that E’s velocity is U,D, where D is the point of 
tangency. 


* “Lower” applies to X in the upper half-plane as sketched. For other X, the choice 
of tangent is fixed by continuity. 
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(a) 
Figure 10.2.2 


Proof. If P plays XU, ($ = 1) it is clear that the resultant velocities 
available to E are those with vectors extending from X to points of the 
circle (such as XF in the figure). None penetrates XD in the downward 
direction. 

If E plays U,D, let U_D_ be the translate of E’s velocity drawn from U_ 
(see (b) of the figure). The resultant velocities available to X are those 
with vectors XG, where G is a point of the closed segment DD_. None 
penetrates XD upwardly. 


LEMMA 10.2.3. The semipermeable surfaces (in the right half-plane and 
oriented as above) are involutes of the circle #,. 
Proof. From X draw the tangent XJ shown in Figure 10.2.3 to #,. 


y 





Figure 10.2.3 
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In the right triangle CJX, the ratio (short leg: hypotenuse) is w,R/w,d 
where d = |CX|. The proof of Lemma 10.2.1 shows that |X’ U| = w,d/R 
so that this ratio is the same for the right triangle U,DX. Thus the 
triangles are similar. As, from Lemma 10.2.1, XU, is perpendicular to 
CX, it follows that semipermeable direction XD is perpendicular to XJ. 
Classical knowledge of direction fields and differential equations now 
establishes the lemma. 


LEMMA 10.2.4. Draw the lower ray from O, which is tangent to %,. 
The involute of Lemma 10.2.3 ceases to be semipermeable below this ray. 
Proof. Referring to Figure 10.2.2b, it is clear that the semipermeable 
property of XD fails should the segment DD_ lie on the same side of XD as 


y 





Figure 10.2.4 


U,. Thus it suffices to show that, as X moves clockwise along the involute, 
the angle D_DX decreases and it becomes zero when X meets the above 
ray. 

Now DD_ is parallel to the headline U,U_ and we recall that the latter 
is perpendicular to OX. Thus we need show that, as the involute unwinds, 
the complementary angle «, between XD and OX (see Figure 10.2.4), 
increases and becomes a right angle when X reaches the ray (such as at B in 
the figure). But this is obviously true, as « steadily approaches a right angle 
as OX and JX approach coincidence; which they attain when X is at B. 


We are now in a position to (re)state the construction of the barrier. We 
can suppose a capture region bounded by an arbitrary convex curve & 
about O. 
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THEOREM 10.2.1. The right barrier is constructed as follows, the left one 
being treated symmetrically: 

Draw the involute, unwinding clockwise, of #4 which contacts & and 
is otherwise exterior to it. If this is impossible, there is no barrier. Other- 
wise the barrier is the arc of this involute extending (in the unwinding 
sense) from the contact point with @ until the first point where it either 
meets the left barrier or the lower ray through O tangent to 4 ,. 


The only detail of the proof not supplied by the preceding lemmas is that 
the useable part of @ is spanned by the contact points of the involutes. 
However, if we assume @ is smooth‘ and accept as known that the useable 
part is a connected arc on the upper part of it, then, as tangency to the 
barrier is equivalent to the BUP condition (Section 8.5.1), all follows. 


10.3. THE PRIMARY SOLUTION OF THE HOMICIDAL 
CHAUFFEUR GAME OF DEGREE 


The term primary applies to the integration of the RPE with initial 
conditions on % to obtain the paths terminating thereon. In most examples 
it is the first integration we do and an important one; in those with no 
inhibiting singular surfaces indeed it yields the full solution. 

The present problem is exceptional in that the primary integration is of 
minor significance. The optimal paths on the right are shown in Figure 





Figure 10.3.1 


$ For general @ (which the barrier may meet at a corner) it is not hard to discuss the 
game of kind in terms of embedding family of semipermeable involutes, somewhat in 
the manner of Section 8.5. II and not explicitly invoke useable parts at all. 
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10.3.1 as well as the curves of constant V. For simplicity, we suppose the 
@ to be a circle centered at O. 

We know that when the initial distance between P and E is appreciably 
large, the chase will always culminate with x on the universal surface on the 
y-axis. In the realistic space we know the interpretation: P follows E 
along a straight path until capture occurs when E meets the foremost point 
(C in the figure) of €. 

The primary paths mark the exception: For E initially close and not too 
much off center, P captures with a sharp turn only. The path of E is 
straight; its optimal direction is such that he enters € perpendicularly in 
the sense of the relative motion of both players. 

It is not hard to show that the envelope principle5 holds: the barrier Z 
is tangent to the curves of constant V, and G itself is an optimal path of 
minimax time of capture. 


Problem 10.3.1. We have stated that the optimal paths enter the useable 
part perpendicularly to @. Yet the barrier, similarly an optimal path, 
meets @ tangentially. In view of continuity of the optimal strategies, 
explain this seeming contradiction. Generalize. 


For convenience, we rewrite the KE: 


è= — Tyg + wasin y 


ý = “2 2d — w, + wy 008 y, -l<¢<l 


where w, = P’s speed 
w = E’s speed 


R = minimal turn radius of P. 
For @ we have 
z = lsins, y = lcoss 


and & is the z,y-plane exterior to this circle. 
The many preceding similar problems will render details of the inte- 
gration unnecessary. We state some results without proof. 
The useable part of @ is 
= Sy SS < Sy (10.3.1) 


where cos sy = w,/w,, sin sy > 0. 


5 Section 8.8. 


Cn ar 
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The optimal paths are given by 


w . w 
x = (l — w7) cos (s+) + Rent 


y = (l — wer) sin (s+) + r(1 — cos“), 


where s satisfies (10.3.1) and 0 < 7 < J/w,. 

The curves of constant V are arcs of circles with centers R(1 — cos (w,/R)7), 
R sin (w,/R) 7 and radii 7 — war. Their envelope is an arc of Z, which is 
also the optimal path with s = 5». 


Exercise 10.3.1. Prove these statements. 


Research Problem 10.3.1. Although Figure 10.3.1 is drawn for the case 
where the barriers do not meet, our reasoning, of course, applies to the 
case where they do. How then does the point A, where the paths coalesce 
(T = I/w,), relate to the point of intersection of the barriers? In other 
words, when the barriers meet and there is but a small bounded capture 
zone, is this filled by primary paths or does a universal curve and its 
tributaries cover part of it? 


10.4. THE UNIVERSAL CURVE AND ITS 
TRIBUTARIES 


We have already seen in Example 7.13.2 that the y-axis above C is a 
universal surface and the only® such. It and its tributaries correspond to the 
most obvious and general type of play. As redepicted in Figure 10.4.1a, 
P first turns sharply as possible until he is pointed at E and then travels 
straight; throughout Æ traverses the same straight path which lies on a 
tangent through his initial position to the nearer of P’s initial curvature 
circles. Such was proved in Example 7.13.2. 


LEMMA 10.4.1. In the domain of the tributaries, the curves of constant V 
are involutes of the pair of turning circles.’ 

(By “pair” we mean that if the involute is drawn in the usual way by an 
enwrapping string, the union of the two circles can serve as the enwrapped 
solid guide as shown in (c) of the figure.) 

Proof. We know that E’s flight direction is along the tangent, here the 
generating “string.” From the ME, we know that his optimal direction is 
normal to the curves of constant V. A simple principle of differential 
equations now confirms our result. 


* An inverted sort of exception will appear in Section 10.9. 
? The circles of centers (+R, 0) and radius R. 
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Figure 10.4.1 


An appendix to this chapter contains some of the analysis relating to the 
domain of the tributary paths. 


Research Problem 10.4.1, Do the optimal paths here—the tributaries to 
the universal curve—admit of such a simple geometric interpretation ? 


The optimal paths are sketched in (b) of the figure. One of them, that 
which passes through B, the barrier end, meets the lower y-axis at F. 
From each point of the axis on or below F two optimal paths, right and 
left, emanate. These points thus constitute a dispersal curve. In the 
realistic space they correspond to positions at which E is directly behind P. 
The simple type of play (as at (a) in the figure) is optimal, but both players 
are confronted with a left-right choice and the problem of out-guessing 
the other. It is a typical instance requiring an instantaneous mixed strategy 
as discussed in Chapter 6. 

On Figure 10.4.1b is indicated all covered thus far of the solution of 
the homicidal chauffeur game when the barriers do not meet. But we are 















































282 EQUIVOCAL SURFACES [10.4] 


challenged by a large empty region of @ untouched as yet by any phase of 
our solution. It is roughly bounded above by Æ and the nonuseable part of 
@ and below by arc of the path BF. From starting points therein—X, on the 
figure is typical—we are intuitively led to believe that a swerve must 
follow. If this is so, for a state such as X,, P must begin with a sharp left 
turn, which will occasion a velocity of X, somewhat as shown by the 
arrow. It seems safe to conclude that ultimately x will cross BF and will be 
in a domain already treated. But what of the transition? This proves to be 
far from a simple question and to it we now turn. 


10.5. EQUIVOCAL SURFACES 


They are singular surfaces of type (+, u, —). In this section we shall 
describe the general conditions for the occurrence of an equivocal surface 
(abbreviated ES), but our later, more refined analysis will be limited to the 
case where n = 2. In fact, a single example will suffice to elucidate the 
whole theory. 

The names derives from the choice of two optimal strategies at each 
point of the surface at the disposal of one of the players. 

These surfaces, unlike many types already studied, have no counterpart 
in the calculus of variations. They cannot occur in a one-player game. 
The implication is that the underlying theory of differential games must 
essentially depart from any mere extension of classical ideas. 

Suppose, in a certain differential game, that it is clear, on intuitive or 
other grounds, that the optimal paths behave as typified in Figure 10.5.1. 
Those that reach @ belong to some family (1), but at (2) there is an entirely 
different family. Optimal play demands that x, starting as shown, must 
first traverse a path (2) and then switch to a path (1). The paths (1) are 
yielded by a valid solution of the main equation which extends well up to 
and beyond where a juncture with paths (2) would be expected to occur. 

We are interested in the mechanism of the transition from one type path 
to the other. 

For convenience we shall refer to the paths (1) as primary and their 
union as the primary domain; the paths (2) will similarly be secondary. 

Let F be the surface at which the transition occurs (on the figure it 
might be one of the dashed curves). On reaching / via a secondary path, 
x must either penetrate S, remain on it, or regress backward. The third 
alternative is absurd if the secondary paths are posited as optimal and the 
bearers of unique optimal strategies in their domain. 

The second alternative, depicted at (a), has acting as a semiuniversal 
surface: once x reaches S, x traverses it. We will refer to values of the 
control variable which engender such play as traversing strategies. 

In the remaining alternative ((b) in the figure), S acts as a transition 
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surface: x penetrates S. We will speak here of penetrating strategies. 

We will now make four explicit assumptions: 

A1. The choice of which surface is to act as / rests entirely with one of 
the players. For definiteness let us say he is P. 

A2. The choice as to whether x penetrates or remains on / rests with 
the other player. Calling him Æ, then he alone chooses between the 
penetrating and traversing strategies. 

Now let us consider some “smooth” one-parameter (A = the parameter) 
family of surfaces, each eligible to be . By “smooth” we mean that 
distinct members of family do not intersect, and as 4 increases, the surfaces 
shift smoothly in the same general direction. (See the set of broken curves 
in the figure.) 

A3. Starting from any fixed x in the secondary domain, let E elect a 
traversing strategy which is optimal in all other aspects at his disposal. 
Let P choose various A (member of our family of possible ) but otherwise 
act optimally. Then the payoff is a decreasing’ function of À. 

A4. Under the same hypotheses, except that now E elects a penetrating 
strategy, the payoff is an increasing® function of A. 


8 If (without changing the parametrization) both of these rates are reversed, the con- 
clusions we soon will draw from these assumptions remain valid. 
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Payoff 





Figure 10.5.2 


Plots of these two payoffs, as functions of 2, have the appearance then 
of Figure 10.5.2. As E is the maximizing player, for each 4 his choice 
between the traversing and penetrating strategies will be that yielding the 
higher payoff. Thus it will be represented by some point on the composite 
upper curve (overscored in the figure). Consequently P will pick the A 
(Ay in the figure) which yields the minimum of this upper composite curve, 
which will be at the intersection. 

We draw the conclusion, as this situation prevails for all smooth 
families and all admissible starting points in the secondary domain, 
that the optimal surface S must enjoy the property: 


From each point of S optimal play of the traversing or penetrating 
type each leads to the same payoff, the common value being the Value. 


This requirement will be known as ES condition and a surface fulfilling 
it, an equivocal surface. 

For n = 2, the only case which we have studied in any detail, it turns 
out that the ES condition is tantamount to an ordinary, first-order dif- 
ferential equation. Thus, generally, an equivocal surface can be passed 
through an arbitrary point. What starting points befit a particular game is 
often clear from its context; generally we might observe a possible analogy 
between this question and that of fitting a barrier (Section 8.5). 

Still retaining n = 2, let P and E have roles as in the assumptions. 
We shall suppose that P has a linear vectogram and his optimal strategies 
in the primary and secondary domains use the two extreme values of ¢. 
(It is hard to see how anything else is possible.) To navigate the equivocal 
surface, P will require an intermediate ¢ = ¢. Further, we shall assume 
that time of capture is the payoff (G = 1) although it would not be hard to 
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extend the following results to arbitrary positive G. We also suppose 
vectogram additivity, that is, the resultant velocity of x is the sum of the 
two players’ choices. 


LEMMA 10.5.1. Under the above circumstances, the optimal traversing 
strategy of E on an equivocal surface® is that which maximizes the velocity 
component perpendicular to P’s headline in a direction counter to that of 
x’s travel on the surface. 

Proof. With G > 0, it is clearly best for E£, when he permits traverse of 
the equivocal surface, to keep the speed along it as low as possible. In 
Figure 10.5.3 let X be a position on the equivocal surface ES, XA, and 
XA, be P’s extreme velocity vectors (414 is his headline), and let the 
dashed line be tangent to ES at X. Whatever be his vectogram, suppose 
that E selects a velocity A,B, (or its translates A,B, or AB). Now ¢ 
must yield a resultant tangent to ES; thus P selects XA, such that the 
resultant XB; lies on the tangent. Then, to minimize |XB,|, E strives to 
make the line B,B, as far from A,A, as possible. But such requires the 
strategy of the lemma. 


When the equivocal surface is known, then so too will be V on it; the 
secondary paths are constructed in the usual way, using this data as initial 
conditions. The following lemma shows that the initial V, are reckoned 
with E playing y, the optimal traversing strategy of the above lemma (and 
of course, P playing one of the extreme values of ¢). 


LEMMA 10.5.2. The optimal traversing strategy of E on an equivocal 
surface is continuous with his optimal strategy on the secondary paths. 

Proof. Suppose the lemma false; let E play his optimal strategy with the 
alleged discontinuity and let P play thus: recalling Al (choice of which 





Figure 10.5.3 


° More appropriately “curve” when n = 2. 
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curve is to act as ES resides with P) let him choose one, Z’, close to the 
actual ES on the “secondary” side. That is, P makes his switch from an 
extreme ¢ to $ slightly before x reaches the equivocal surface. He can do 
so with (at worst) an arbitrarily small penalty in payoff. 

But as 2” lies in the secondary domain, due to the discontinuity, E’s 
opposition to the travel of x along S” will be bounded away from the 
optimum of Lemma 10.5.1. Thus, if P’s defection above is sufficiently 
small, he will obtain a payoff better than the Value. But such is impossible 
against an opposing optimal strategy. 


Problem 10.5.1. Give a formal proof according to the following outline. 
For the KE take 
&; = a(x) + BAY, x), (i = 1, 2). 
The initial conditions for the V; on the ES (parametrized by 7) result from 
lvi 4 < ge 
SOVS Vlux)d; + BG, 0] = —1 (ES condition) 
t=152 
and & > Via; + max > Vf: = —1 (ME). 
i v 


Show that a solution can be obtained through 
YaV, = 0 
i 
È BAY, OV; = max $ BV, = —1. 
? Y i 


(To apply Lemma 10.5.1, observe that Ba, — faa, is the requisite normal 
velocity.) l 


10.6. AN EXAMPLE WITH AN EQUIVOCAL 
SURFACE: PRELIMINARIES 


Can a differential game actually have an equivocal surface as part of its 
solution? If so, in a case such as sketched in Figure 10.5.1, then part of the 
primary solution would have to be abruptly cut off even though the reject 
is part of a formally valid construction. This matter merits close scrutiny 
which we bestow on this aspect of the example below. 

A second dividend accrues from study of this example. It emulates 
that part of the homicidal chauffeur concerning passage around the end 
of the barrier. It does so closely enough to warrant our applying the 
conclusions drawn here directly to our main subject game. 


Example 10.6.1. A game with an equivocal surface. We take & to be 
the upper half-plane (y > 0) and @ to be the positive z-axis: 


€: x=s2>0, y=0 
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The payoff will be time to termination. The vectogram for P will be as in 
Figure 10.6.1. The vertical component will be bounded by the constants 
+b, whereas the horizontal component will be u(y), a smooth, increasing, 
and positive function. On the other hand, E will have a circular vectogram 
of fixed radius w. We shall require that b > w > u(0) and that for some, 
necessarily unique, yo 

w = U(Yo). 


From Example 8.4.3 we know the barrier Z. It is shown as the arc OB 
in Figure 10.6.1b, the coordinates of B being zp, Yg with yg = yp. Note 
that at B, Z is vertical; the barrier terminates just as in the homicidal 
chauffeur game—when its tangent is parallel to P’s headline. 

The least time paths, when starting positions are rightward enough to 
permit unhindered access to @, are easy. Since all vectors are independent 
of x, P and E simply maximize and minimize their downward vertical 
velocity components. That is, P selects his lower extreme vector and E’s 


i 
4 


o=-1 


u(y) 
ġ=1 
(a) 
y 
L 
B 
x 
9 @ 
(b) 


Figure 10.6.1 
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L ġ=i1 





$=-1 





Figure 10.6.2 


velocity points straight up. The resulting paths are all translates of the same 
curve and are sketched at (b) of the figure. Let L be the arc of that one 
passing through B which lies left of B. Those paths which are not ter- 
minated by Æ constitute the primary paths in the sense of the last section. 
In the primary domain it is clear that (consider the vertical velocity 
component), 


Veo, (10.6.1) 





Exercise 10.6.1. Calculate the full solution in the primary domain when 
b=3,u=y+lw=2 10 


At this point one might suspect that the solution could be completed by 
taking initial conditions on L and obtaining the optimal paths which feed 
into it from below. That is L is to be a semiuniversal curve and the optimal 
paths to appear as in Figure 10.6.2a. 

Such a construction can actually be carried out. On L, H is to be the 
primary V as given by (10.6.1), and P selects his upper extreme velocity 
vector. The analysis is not difficult. 


Problem 10.6.1. Prove that a solution as in Figure 10.6.2a can be con- 
structed and/or actually construct it for the specific data of Exercise 10.6.1. 


THEOREM 10.6.1. The preceding construction does not supply the correct 
solution. 

Proof. Suppose it did. Then P’s optimal ¢ = $ would entail the choice 
of his lowermost velocity vector for x on or above L and his uppermost 
when below. On L, as we have seen, the optimal y = @ leads to a vector 
pointing straight up for E. 


10 The barrier with this same data was the subject of Exercise 8.4.3. 





[10.6] AN EXAMPLE WITH AN EQUIVOCAL SURFACE 289 


Diagrammed, these strategies appear as in Figure 10.6.2b, where X is a 
point of L. The above strategies lead to the velocities XA for P and AC for 
E. Thus the resultant XC is tangent to L. 

We will now have E play an alternate strategy. In some neighborhood 
N of L, E chooses a velocity vector pointing due left, which appears as A D 
in the figure. Let P play a K-strategy (Section 2.6) with ¢ as tactic and 
finely enough subdivided so that all his points of decision fall within N. 

For X on L, P will thus choose XA. The resultant is now XD, which 
brings x (or X) below L. Thus P’s next decision will invoke the upper 
vector XA, yielding XD, as resultant velocity. Ultimately x will again be 
brought on or above L and the resulting velocity returns to XD. 

Thus x, by an alternation between velocities XD and XD,, follows a 
path oscillating about L. With finer subdivisions, such, it is easy to see, 
approaches an equivalent motion with velocity YF,, the convex linear 
mixture of XD and XD, which is tangent to L. That is, the payoff becomes 
as if P had played the intermediate ¢ resulting in velocity XF. (FF, is 
horizontal.) 

But |XF,| < |XC|. Thus x traverses L more slowly under the alternate 
strategy and the payoff is increased. Thus the type of play pertaining to 
(a) of the figure is not optimal, for E can do better than the payoff it 
yields when opposed by P playing all sufficiently fine K-strategies with ¢ 
as tactic. !! 


COROLLARY 10.6.1. Not all of the primary solution is valid. 

Proof. If the “solution” above L were correct, we could formulate a 
new game for which & is bounded by &, L, and the negative y-axis. Here 
and L are to form @, the usable part of which is clearly L. Further, H 
is to be the primary Value on L while G is again 1. The actual solution to 
this new game results from the construction whose existence we asserted 
(and requested in Problem 10.6.1). Then the solution of the original game 
would have to be just what we proved it was not. 


Before continuing with this example, let us observe that essentially the 
foregoing construction is valid in similar one-player games. 


Example 10.6.2. The same as Example 10.6.1, except that now w = 0. 
The KE are now ; 
& = u(y) 

y= bd, —-1<¢<l. 

There is now no barrier. But L persists; it is now the path of the first 
stage solution leading to the origin. Adapting the preceding ideas, we are 
led to a construction having paths as at (a) of Figure 10.6.3. In view of 

^ Such a strategy appears to be the only way we can assign a meaning to his 
playing ¢. 
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g=+1 


(0) 





(a) (b) 
Figure 10.6.3 


Theorem 10.6.1, we give a brief but direct proof that the constructed 
solution is correct. 

Clearly P should play his lower extreme velocity (¢ = 1) throughout the 
whole partie whenever this brings x to @. For we can reckon the payoff in 
terms of the vertical velocity component only, that is, consider the time 
required for the projection of x on the y-axis to descend to @. But ¢ = 1 
yields the maximal possible downward speed throughout play and there- 
fore is certainly best. Thus the primary stage paths are correct. 

Suppose now x is to travel from a given point A to another B to the 
right of A but too distant to admit the preceding strategy. The path of 
least time is depicted at (b); the first part has é = —1 anda second ¢ = 1. 
Exactly one such path exists, for if we draw the 6 = —1 path through A and 
the retrograde path with ¢ = 1 through B, they will intersect just once. 
To see that this path is optimal, we think this time in terms of the hori- 
zontal velocity component. It is greater with increasing y and clearly the 
path at (b) attains the maximal integrated y. 

Thus if the starting point lies to the left of L, the least time path to O is of 
the type asserted and shown at (a). Finally, it cannot be that P can do 
better if x first reaches @ at some point rightward of O. For were this the 
case, the path would have to cross L; at the first such instant it is under 
the aegis of the primary strategy and follows L to O. 


Problem 10.6.2. Show that if u = y — 1, and so not always positive, that 
not all of L acts as a seat of initial conditions for the second stage paths. 


Problem 10.6.3. Still with u = y — 1, if £ is enlarged to the entire plane, 
but @, save possibly at O, must be approached from above, show that 
there are “third stage” paths. 


10.7. AN EXAMPLE WITH AN EQUIVOCAL 
SURFACE; SOLUTION 


We return now to Example 10.6.1. We are going to show that the 
assumptions Al to A4 of Section 10.5 are satisfied. As a prelude we 
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observe that some parts of the first stage paths are valid; on the other 
hand, in some region below and left of B the optimal $ must be —1.12 
Our problem is the transition between these phases. 

Al. Clear. For P can pick the curve which demarcates the regions 
where he takes ġ = 1 and ¢ = —1. If this curve is to function as an 
equivocal one, of course he must use an intermediate ¢= $ on it. But 
from the standpoint of K-strategies, this is unnecessary. As we will see 
under A2 below, the ES-strategy of E compels a zigzagging about the ES 
somewhat as in the proof of Theorem 10.6.1. 

A2. The curve that we shall refer to as the ES in this paragraph can be 
any smooth one. such that 


1. it starts at B. 

2. it cuts each primary path above B with steeper slope so that the 
equivocation in the definition of an ES is possible. 

3. it can be navigated by x when E plays #, the optimal traversing 


strategy, and P plays some intermediate ¢ which we shall call $. 


We will show that E dictates the choice between traversing and pene- 
trating play. 

The various possibilities for a vectogram for X, a point on the ES are 
shown in Figure 10.7.1. From 2, X, is above B and so u > w. Here 
A,A; is P’s headline. As the optimal primary (and also penetration) 
strategy calls for E’s erect velocity vector A,A, and P’s primary optimum 
is $ = 1 or the vector X,A,, the resultant velocity XA, is tangent to the 
primary path. From 2 the tangent to the ES has a steeper slope and is 
shown as the dashed line. From Lemma 10.5.1, the optimal traversing 
strategy of E yields the vector A,B, (or AgB, or A,B,) perpendicular to 
A,A,. Thus $ leads to the vector X,A, by P so that the resultant X,B, is 
tangent to the ES. 

Now suppose E should switch to 4,4, (penetration strategy). The 
resultant velocity choice left to P ranges from X,A, to X,A;. All carry x 
above the ES into the primary domain. From here, if each player wants to 
reap the Value, he must play optimally and X,A, results. Thus the partie 
concludes in the primary domain. 

Now suppose E adheres to the traversing strategy A,B,. We will show 
that P cannot deviate from $ without incurring a loss in payoff. 

Suppose for some short interval he used a $ < ĝ. Then x would rise 
a positive distance above the ES, and the primary optimal strategies would 
prevail for both players. By thinking of vertical velocity components only, 
we see that this route, where x rises and then descends again, takes more 


12 Upper extreme velocity. 
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Figure 10.7.1 


time than a direct primary descent from X, and therefore more time than 
V(X). Z 

On the other hand, if P adheres to a ¢ > ¢ during some interval, x 
descends below the ES. (By Lemma 10.5.2, E keeps close to the velocity 
A,B,.) At some later time x must recross the ES, say at X,. For, by 1, the 
ES and Æ form one curve separating x, after its descent, from @ and E can 
prevent x from crossing Z. Let E adopt a strategy which does so prevent, 
yet which otherwise retains A,B, when x is below the ES. Now the 
depressed route from X, to X, takes longer than the one via the ES. We 
can see this by considering horizontal velocity components only and 
recalling that u(y) is increasing. Thus there is a way for E to force P to do 
worse than the Value. 


To deal with A3 and A4 we take for our possible Z a family of curves 
each fulfilling 1, 2, 3 as described under A2. Let A be as in Figure 10.7.2a. 

A3. We are to show that, when E plays the traversing strategy, the 
payoff decreases with increasing A. It suffices to show that if (see (b) of the 
figure) AB and DB are arcs of Z curves, the time for x to travel ADB is 
less than that for travel over the curve AB. Of course, AD is part of a 
“secondary” optimal path. 

We may concentrate on horizontal velocity components. On AB and 
DB such are u(y)—w and for AD it > u(y)—w. As u increases with y, the 
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higher path ADB everywhere engenders the greater horizontal velocity and 
so consumes less time of traverse. Thus the payoff decreases with A. 

A4. We now show that if x penetrates ¥ to the primary domain, the 
payoff increases with A. From (10.6.1), V is an increasing function of y 
in the domain. Thus, as the Z curves have negative slope, the greater A, 
the more time for x to reach S and the greater V once x gets there. 


Thus the existence of an equivocal surface is established. We now turn 
to its construction. The ES condition states generally that for travel 
along this surface 


Z = —G(= —P) (10.7.1) 


where Vis the Value in the primary domain. In our example we shall work 
retrogressively, G = 1, and V is given by (10.6.1). But first we must write 


the KE: 
& = u(y) + wsiny 





ý = —bd + w cosy, -l<¢d<l. 
From Lemma 10.5.1, sin y% = —1, cos y = 0, and (10.7.1) becomes in our 
case 
o o Y 
1 =P= vR + Vi =0+— vf. (10.7.2) 
b—w 
Thus for the ES 
» w 
¢=1— (10.7.3) 
which, when inserted in the KE, furnish its differential equations 
& = —u(y) +w 
ee eee (10.7.4) 
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Figure 10.7.2 
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Our equivocal curve is the integral of this system of differential equations 
beginning at B: for initial conditions take x = zp, Y = Yg = Yọ When 
T = Tg = V(B) = yYo/(b — w) from (10.6.1). Thus 7 on the ES is F. 

Note that at B, z = 0, y > 0 so that the equivocal curve here has a 
vertical tangent. We learned in Example 8.4.3 that the same was true of the 
barrier. Thus the two curves join smoothly at B. 


Research Problem 10.7.1. Is this a general truth? When an equivocal 
surface attaches to the termination of a barrier, do the two surfaces meet 
smoothly ? 


Exercise 10.7.1. Using the data of Exercise 10.6.1, show that in this case 
the ES is a half-parabola. 
[In fact, its equations are 
x = —}(T — 1)? + zp 


y=r 


(10.7.5) 


frr>1] ` 


We conclude this section by partially treating the secondary paths— 
those emanating from”? the equivocal surface— with an eye to corroborating 
the material of Lemma 10.5.2. 

The ME, of the current problem is 


u(y)V, — BBV, + wp +1=0 
where 


$ = sgn V, and also sin Ẹ = V,/p, cos § = V,/p, p = VV + V,2. 


The RPE are 
è= -ua - w2, ¥,=0 
P 
o V, 3 f 
TE Vy = u'(y)V,. 


For initial conditions we use the equations of the ES with, for notational 
consistency, 7 replaced by s. Then on the ES, V = s. Using (10.7.4) for 
z, and y, our usual 
Vats + Vy 8 = V; 
becomes in this case 
—V.(u — w) + V(b — w)=1 (10.7.6) 


which is to be solved for V, and V, in conjunction with the ME,. We can 


18 In the retrograde sense. 
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Figure 10.7.3 


see at once that a suitable formal solution is 


—1 
V, = PERE, V, = 0. (10.7.7) 

We must note that V, < 0 when y > Yo or on all the ES but B. 

To ascertain 4, we must look at V, on the ES. From the RPE 

V,=u'V, <0 

as u’ > 0, V, <0. Thus $ = —1 or P uses his uppermost vector as we 
should expect him to. 

From (10.7.7) we see that initially (r = 0) sin y= —1. Thus £’s 
velocity is left and horizontal; it is continuous with the ES strategy as 
Lemma 10.5.2 asserts. 


The remainder of the problem entails little innovation and we leave it 
to the reader. 





Exercise 10.7.2. For the data of Exercise 10.6.1, show that the optimal 
tributary paths to the ES are given by 


z= (1 — s)r + $e? — JI + 7? + log (r + V1 + 7) — Mo — 1)? + zp 
y = —3r + NWI F 2s. 
Finally, the picture of the optimal paths is something like Figure 10.7.3. 


10.8. DISCUSSION OF EQUIVOCAL SURFACES 


Can the equivocal surface, as a smooth adjunction to the barrier, be 
regarded as an extension of the latter? Only in the exiguous sense that each 
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player can make x penetrate the surface only at the cost of a penalty in 
payoff. 

It is interesting to see how the players would be bound by such pro- 
scriptions in practice, that is, in some realistic game for which an adequate 
model is like the present example. 

Suppose at a certain point of an (optimal) partie, x reaches the equivocal 
surface and E elects the traversing strategy. Continued optimal behavior 
by P retains the ES as a path and ultimately x reaches B. Here E no 
longer has the dichotomy of strategies; to maximize the payoff he is 
bound to the primary choice. But also at his disposal is the optimal strategy 
of the game of kind. If he plays so, then P must retaliate with his such 
strategy or else x will penetrate the barrier and P will not be able to compel 
an imminent termination. To obtain a later one entails recrossing of the 
ES; if E repeats the tactic and P the response, continued recurrence pre- 
cludes termination ever. Thus P is forced, from B on, to play the game of 
kind strategy; x follows @ and he must be content with a neutral14 
outcome. 

Is it wise for E to act so—to play the game of kind optimal strategy when 
x reaches B? It depends on how we assess the neutral outcome. We could 
adopt the point of view of Chapter 8 and consider neutrality as something 
inferior to proper interior termination. To have an exact theory we must, 
of course, modify the original game by specifying a numerical value of the 
payoff for the neutral outcome. How we so so determines the answer to 
the above question. 

In a “practical” execution the neutral outcome lies at the brink of 
failure to terminate; such can be occasioned by any slight accident and 
P will covet a margin of safety. The simplest and most reasonable way to 
attain it seems to be: when x first reaches the ES, P continues his strategy 
(¢ = —1) for some short interval, thus bringing x across the ES, slightly 
into the legitimate primary domain. 

An arbitrarily small penalty of payoff now buys P release from the 
dichotomy of the ES, for from now on there is single straightforward 
optimal strategy for each player.t® 

Thus, interesting as the ES phenomenon is in theory, the accompanying 
action probably is not of much importance in applications. For in 
realistic play it will probably be circumvented as above. But the equivocal 
surface remains as a boundary, delineating two different kinds of optimal 
play. 

14 As in Chapter 8. Here the meaning is that æ touches @ at its endpoint, the origin. 

15 Let us note that “realistically” P was liable to such a penalty even under optimal 


play. For if switches from the traversing tothe penetrating strategy at some unexpected 
instant, in “practice” there is bound to be some positive time lag in P’s response. 
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We conclude with 


Research Problem 12.8.2. If u(y) has a maximum, say at y,, then it is clear 
that the line y = y, will bea ¢-universal surface sufficiently far to the left. 
What is the rest of the solution? In particular, do the US and ES (if there 
is one) meet? 


10.9. THE EQUIVOCAL PHENOMENON IN THE 
HOMICIDAL CHAUFFEUR GAME 


The ideas of the preceding sections expeditiously fit the homicidal 
chauffeur problem; we can construct an equivocal curve, which begins at 
the end of the barrier, by the same method. 

Let us suppose the left and right barriers do not meet and that x starts 
from a point near and under the right barrier. Capture requires the 
Swerve maneuver: P must force x downward and around the barrier. He 
begins by sharply turning left (6 = —1) and so occasions the descent of x. 
But at some later time he will be playing the strategy of the right trib- 
utaries of the universal surface, which call for $ = 1, that is, he turns 
sharp right until E is directly in front of him (x on the US) whence the play 
ends with a direct chase along a straight line. The ES is here the locus on 
which P makes the transition of $ from —1 to +1. 

On the ES, E has the choice between the traversing and penetrating 
strategies. What is #, the value pertaining to the former? From Lemma 
10.5.1, we see that # demands that E point his velocity vector normal to 
P’s headline. From Lemma 10.2.1, this headline is normal to the radius 
vector OX. Thus E’s velocity directly toward O, implying 


In the realistic space if E plays the traversing strategy he follows a 
course of pure pursuit’? of P. 
As for P, his optimal strategy ¢ for x on the ES is generally distinct from 
0 and +1. This appears to be the only optimal instance of his following a 
path other than straight or circular with radius R. 
The procedure for ascertaining the ES parallels that of Section 10.7 and 
we need not repeat the reasoning. As above, y is known as function of 2, 
y; we wish similar knowledge of 4. It results from the ES condition 


èV, + Vij = —1 (10.9.1) 


where Ve and V, pertain to the (right) tributary paths and ż, ġ mean their 
expressions from the KE with y replaced by the above ğ(z, y). If then 
(10.9.1) is solved for ¢, the solution will be the sought gle, y). Insertion of 
these values of the control variables into the KE yields a pair of ordinary 


1 That is, E always travels directly toward P. 
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(b) 


E 
Figure 10.9.1 


differential equations; their integral path which passes through B, the 
endpoint of the barrier, is the ES. , , 

The foregoing analysis is done in detail in the appendix to this chapter; 
the final differential equations appear as (10A.9). They seem to be par- 
ticularly intransigent and we have unearthed little of their specific geomet- 
ric properties. , , 

Figure 10.9.1a is an accurate construction of an instance of the equivocal 
curve; at (b) are similarly drawn the corresponding paths of P and E in 
the realistic space during this phase of the partie. 

There are two possibilities for the form of the ES, depending on the 
parameters. It may meet the y-axis—the arc BC in Figure 10.9.2a—or it 
may terminate on @ as at (b). We will discuss only the former case, 
which seems the more interesting. 
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Let us trace an optimally played partie with x starting at X as in (a). 
The optimal path meets, as shown, the y-axis, which here again acts as a 
universal surface. After x reaches it at A, x traverses it to C. Of course, 
other starting positions, such as X}, lead to paths such as X,D which 
encounter the ES directly. 

It is not hard to interpret the path XAC in the realistic space. The 
players start from P and E of Figure 10.9.3; P begins with a sharp left 
turn (radius R) while E travels straight along the tangent EF to the left 
steering circle. After reaching F, P too travels the line EF. Such positions 
correspond to X on the universal AC. Note the perversion of the simpler 
capture depicted in Figure 10.4.1a; now E is doing the pursuing. 

Both continue along the line EF, the distance between them increasing, 
until it equals OC, a fixed constant of the game, of Figure 10.9.2a, The 
positions are now P, and E, on Figure 10.9.3. 

At this point P switches to his “equivocal” strategy $, and he flies the 
curved course P,P, (see Figure 10.9.1b) while E, if he plays the traversing 
strategy, follows along EE; in pure pursuit. Now x is on the ES; at any 
time E may switch to his (here right) tributary strategy. That is, he will 
head along the proper tangent to P’s right steering circle, flying away from 
the point of tangency, etc. 

Note that when he was at P}, P had the option of turning either right or 
left (x could follow the ES on either the right or left side of the y-axis). 
If E responds with the traversing strategy he points at P in either case ; 
there is no discontinuity in y and P is not obliged to mix his two choices. 
But if E should elect the penetrating strategy, he too has a right-left choice 





Figure 10.9.2 
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Figure 10.9.3 


and an instantaneous mixed strategy is called for. Because it is safe for P 
to mix in either case, it should be incorporated into his optimal strategy. 
Recall that the part of the y-axis below C is a dispersal surface and, as 
mixing is required at each of these points, demanding it at C too is not an 
undue strain. 

If E perseveres in the traversing strategy, x follows the ES to B, the 
endpoint of the barrier. Here, for maximal capture time, E must pick the 
penetrating strategy. But he also can play the optimal strategy of the 
game of kind and achieve a neutral outcome. In fact, all the discussion 
of Section 10.8 applies here. As there, in “practice” it would appear 
beneficial for P to allow x to pass slightly below the ES and avoid the 
quandary at a slight cost of payoff. In our description a convenient time 
to do so occurs at the positions P, and £,; P merely continues along the 
straight course EF until the distance P,E, slightly exceeds OC of Figure 
10.9.2(a). 

Finally, we ask if our solution is really complete. Is it possible that 
there can be still another phase to an optimal partie? 

In Figure 10.9.4 are sketched the optimal paths prior to the ES phase in 
a game such as we have just discussed. Can there be a region, such as is 
shaded in the figure, still unaccounted for? 
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Figure 10.9.4 


Research Problem 10.9.1. Answer this question. If “yes,” what is the 
solution in the shaded area? Does there exist a further ES there, so the P 
begins with a sharp right turn to get x on it? 


APPENDIX 


Analytic details 

We shall derive here the analytic expressions germane to the domain of 
the right tributary paths and the differential equations of the equivocal 
curve. We start by collecting the usual material. 


The KE: 
t = —cyp + wasin y 
ý = cxd — wy + w cos y, -l<¢<l 
where w; > w and c = w,/R. 
The ME,: 


—Ach — wV, + p =0 
where A = yV, — xV,, $ = sgn A, 

p=NV + VR, sing = &, cos p=. 
The RPE: á 7 
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We can observe in passing the readily computable result 


A = WV. (10A.1) 


The universal curve is given by 
z=0, y=s, spl 


and as we know F on it, 


(s— J) 


y= 


(wy — We) 
in our usual way, we complete the initial conditions with 


1 


V, = V, = — 
d (wy — Wa) 


and P V, = 0 


the latter following from symmetry and the known continuity of the V, on 
a linear universal surface. On the right we clearly take ¢ = 1. 
Integrating the RPE with these conditions yields 


cos cr (10A.2) 





1 
V, = sin cr, V, = 
Wi — We Wy — We 


x = (s — wer) sin cr + R(1 — cos cr) (10A.3) 





= (s — wat) cos cr + R sin cr 


the latter pair being the equations of the right tributaries. 
Multiplying (10A.3) by cos er and —sin cr and adding leads to 


(x — R)coscr — ysiner + R=0 


which is essentially an algebraic equation for, say, cos cr. We solve it using 
the continuous root such that r = 0 when x = 0, giving us 


—R(x — R) + yh 
a 
(10A.4) 
yR + (z — R)h 
d 


where d = y(x — R} + y? = distance x to (0, R) 
and h = Jd? — R = Vz? + y* — 2cR = length of tangent from x to 
right curvature circle. Then, as we know 7 in terms of x and y, from 


cos ct = 


sin cr = 
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(10A.3) we can easily compute s and then find V from 


v=] +r. (10A.5) 
Wi — We 


To study the ES, direct expressions for V, and V, are more pertinent; 
they follow at once from (10A.2) and (10A.4). 

We know that on the ES, Ẹ points his velocity toward O. Working 
retrogressively, we have E travelling directly away so that, from the KE, 


i= cyp + Wa 
r 


T ` (10A.6) 
y= —cxd + w, + we. 
r 
where r = Vz + y?, 
The retrograde form of the ES condition demands that 
ZV, + yV, =1 (10A.7) 


where the V, and V, pertain to the tributary paths. We substitute in (10A.7) 
our now known expressions for them and also make replacements from 


(10A.6). The resulting equation may be solved for $, and we find after a 
short calculation 
_ War? — eR + rh 


10A.8 
rrp) Vo) 


bet 
where r = Vx? +y, h= Vr — 2zR. 
Thus we know P’s optimal strategy for the ES at each point of & for 


which the constraints on ¢ are not violated. 
The differential equations for the ES are obtained by substituting from 
(10A.8) into (10A.6). They turn out fo be 


Wi 4 Wa 2R — yr — yh 
R R r— y 


We yR + (x — R)r + xh 
R r—y i 


(10A.9) 
o Wy 

= = — — R 
y R (x — R) + 


The initial condition is that the integral curve is to pass through the 
point B. This point we know to lie on the lower ray from the origin 
tangent of .#°,. Now if we hold fixed all other parameters in the problem, 
but alter the size of capture region, the point B can be made to be anywhere, 
within certain limits, on the ray. Therefore for general initial conditions 
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of (10A.9), we can take, when 7 = 0, a general point of the ray, which is 


z= s 1 — (wa/w:}, y = —swe/W1, 


where s > 0, in fact, s is to be reasonably large. 

Research Problem 10.4.1. Solve the differential equations (10A.9) in 
closed form, if this is possible. Do these equations yield any interesting 
geometric properties of the ES? 


Research Problem 10A.2. Whatis the path of P, or the differential equations 
thereof, in the realistic space when x traverses the ES? 





CHAPTER 11 


The Application to Warfare 


In certain cases, such as pursuit games, it is evident how to fit theory to 
practice. But in problems of broader scope, such as those dealing with 
battles or combat, the transition is far from obvious. The present chapter 
aims at clarifying such matters, explaining in general the relations between 
game theory and warfare, and assessing both the utility and the drawbacks. 
Two examples are discussed, analyzed, and partially solved; they appear 
both as worthy problems in themselves and as illustrations of the general 
ideas. 

The first is the War of Attrition and Attack, concerned with the best 
allocation of weapons between these ends during a protracted war. 

The second we have called the Battle of Bunker Hill. The central 
problem is the optimal allocation of firepower when two antagonists are 
nearing one another, the effectiveness of their weapons accordingly rising. 
(The title is, of course, suggested by Warren’s historic command.) 


11.1. GAME THEORY AND WAR 


The treatment of problems of military combat by the theory of (dif- 
ferential and otherwise) games is a big subject and a difficult one. It is in 
fact large enough to warrant its own book, and this single chapter will 
necessarily be but the lightest of sketches. It is difficult because of the 
inference caried by the term game. 

In the present chapter, the application of game theory to a problem of 
warfare will signify that both sides are to make effective decisions;! the 


1 Of course, in other contexts the set of decisions of one side may be null. Thus 
one-player games are really programming problems, and there is nothing wrong in 
treating the latter by the techniques of the former. In fact, a large proportion of the 
problems of applied mathematics can be viewed as one-player games. 
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objective is to determine the Value and the optimal strategies of each (in 
the sense of game theory). A symmetry of approach is basic to this out- 
look; a question of the best method of defense is just as much one of the 
best method of attack. This interrelationship between alternatives facing 
the two players—the decisions of each reckoned on the enemy’s counter 
decisions—is the source of the increased difficulty over the simple (one- 
player) optimizing problem. And the increase may be vast! 

Game theory is a mathematical discipline concerned with problems of 
conflict. Warfare, whose essence is conflict, must ultimately come under 
its aegis. That such has not as yet occurred to a substantial extent 
is due chiefly to the above increase of difficulty and the consequent 
dearth of methods for getting answers. To this point we return in the 
next section. 

From now on we shall restrict our attention to zero-sum two-player 
games. Of course, war itself, as history shows, is almost certain to be 
two-player—alliances always have made it so, ultimately, if not initially— 
but, on the other hand, it surely is not zero-sum. However, we feel that 
war in the large is too vast a subject for the analyst’s pencil save possibly 
in the remote future. But many of its less broad constituents, although not 
exactly so, can be depicted as zero-sum games with adequate versimilitude. 
For example, if we isolate an attack-defense situation, the number of 
weapons (bombers, missiles, tanks, troops, torpedoes) which successfully 
penetrate the defenses is a quantity which the attacker seeks to maximize 
and the defender to minimize. To ascertain the best tactics of each player 
and how much penetration (the Value) accrues therefrom seems a problem 
that may be unobjectionably modeled as a zero-sum game. In fact, the 
adaptability of a situation to a zero-sum approximation may be a good 
shibboleth for mathematical tractability. 

Not every military analysis requires formulation as a game. There are 
some fine ones extant that furnish useful and valuable conclusions without 
the dichotomy of standpoint which is the heart of game theory. But in 
many others the embracing of a two-sided conflict is indispensable. Yet 
the analyst is frustrated by lack of methods. What is available to him, now 
and potentially? 


11.2. THE AVAILABLE TECHNIQUES 


The practical possibilities seem to be three. 


Discrete matrix games 

Such are the stuff of what now may be called the classical theory of 
games. The concept of a game matrix, expounded in any text on the 
theory, is fundamental; it was the means that enabled von Neumann to 
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prove the existence of optimal strategies, pure or mixed, without which the 
entire subject could not exist. 

In principle, every finite, discrete game, as well as many other types, can 
be cast in the matrix form. In practice, the trouble is that the dimensions 
of the matrix will be astronomical unless the game is extremely simple.? 
The simplicity usually means that effectively the game is but one move 
long.* While, in theory, a sequence of moves can be embodied into a 
single strategy, the vast number of such is just what makes the matrix 
colossally unwieldy. 

But there are utilitarian military instances, for example, the distribution 
of limited defense means over a finite set of targets of various values, with 
the enemy countering with an allocation of its attack strength over the 
same set. A second example concerns decoy attacks which may precede 
an authentic one.* The attacker is confronted with the mixed strategy of 
deciding on the number of feints; the defender, on allocating his munitions 
among the apparent raids. 


Differential games 

When long or continuous sequences of decisions confront the partici- 
pants, a practicable mathematical solution is virtually impossible unless 
the game possesses an innate logical coherence in the sense we endeavored 
to explain in Chapter 3. Differential games is the theory of games with 
such a coherence. Being our subject, further discussion will fill the 
ensuing sections. 

The greatest limitation of differential games at present to resolving 
military problems is its restriction to games of complete information. 
Possible remedial techniques and promising, if incipient, ideas are the 
subject of the next chapter. 


Simulation 

Endeavors to play table models of war games, with teams of human 
opponents, often assisted by computers, have been numerous and prolific 
in the recent (as well as the remote) past. This is not the place for a 
lengthy dissertation on so extensive a subject. From our point of view 
experiment is a means both of confirming theory and suggesting new 
avenues for analysis in warfare, as in other sciences. But we must not eye 
the parallel and lose the distinction. 
As we shall stress below in regard to theory, war analysis is much looser 
in laws, predictability, and logic than the physical sciences. For this 


> Tic-tac-toe (tit-tat-toe, oughts and crosses, three-in-a-row) for example, literally 
requires a matrix of more than 10%% rows! 

* There are numerous examples in Reference [8]. 

* Reference [9]. 
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reason simulations with elaborate and far reachingly realistic details 
cannot yield reliable general truths unless the partie is repeated a great 
many times. From the standpoint of differential games, one thing we 
might hope for is confirmation of conclusions of the theory. This is 
especially true when these conclusions spring from simplified models 
(such must necessarily be the case nearly always). How will they fare 
when imbedded in a miasma of realistic detail? 

A second avenue we would advocate lies at the other extreme of playing 
very simple operational games designed to illuminate some specific point 
that has proved baffling to theory. Such, at present, will appear most 
likely in games of imperfect information, such as Example 12.2.4 in the 


next chapter. 


11.3. TYPES OF APPLICATIONS 


In certain cases the role of differential games in problems of warfare is 
explicit and does not require much comment. Such is true, for example, of 
most of the models involving pursuit, evasion, and similar maneuvering. 
We have already mentioned that a strategy is the logical equivalent to the 
scheme of a guidance mechanism, in the sense that 
is supplies instructions as to how to set the controls 
for each set of data measured. 

The utility of the results can transcend a mere 
application of the formal solution. For example, 
if the optimal guidance scheme (strategy) is too 
complex for practical mechanization, we would be 
interested in knowing how good a substitute is a 
simpler one. We could pit it against the optimal 
strategy® of the opponent and ascertain the penalty. 

Additional instances have been noted earlier, 
such as applying the deadline game (Section 9.6) to 
channel patrol or patrol line spacing. Another is 

Figure 11.3.1 the question of whether or how much of launching 
boost is needed by an interceptor missile with a 

faster target. For a case, say, as that of the isotropic rocket (Section 9.3), 
if the barriers meet, the quarry cannot always be caught. But a cross 
section of the barrier of constant v(= P’s speed) supplies the set of target 
positions where capture is certain, no matter how the latter maneuvers, 
when v is the missile launch speed. The section will appear as in Figure 
11.3.1. If the missile is aimed—as it should be—directly at the target, only 


Range 


5 Or, sometimes better, against a strategy deliberately designed to defeat the simple 
one. 
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the range, as shown in figure, matters. But such is the height of the crest 


and calculated from the simple formula derived from (9.3.8): 
— fw? — 
Range = EE 


11.4. THE BROADER PROBLEMS OF COMBAT 


We feel that one of the most fertile and profitable contributions of 
differential games to warfare lies in a broad general category of battles 
and combat. The path from mathematical results to usable knowledge is 
not a simple, direct one, and we shall endeavor below to offer guidance 
for its traversal. 

We shall illustrate the type of ideas in mind by two examples; both 
shall continue in this role as we go into further detail later on. A version 
of the first has already appeared—Example 5.4, the war of attrition and 
attack—but we shall soon try to examine its central question in realistic 
terms. In a long-term war how best should a side apportion its attacks 
between the long-range objective of destroying the enemy’s source of 
supply and the short-range one of engaging his weapons in conflict 
directly? 

The second we have called the Battle of Bunker Hill because our 
central question brings to mind General Warren’s famous, “Don’t fire 
‘til you see the whites of their eyes!” When forces of any type are ap- 
proaching one another, the effectiveness of their weapons increasing with 
the lessening distance, how best should each distribute (or conserve) his 
firepower? Too early action is wasteful because of the small hit pro- 
bability; too late action risks the enemy’s firing first and decimating too 
much potential fire before it is put into action. 

How can such broad questions as the foregoing two be answered? 
Note that they are permeated by the basic concept of game theory: each 
side has choices to make and the merits of his decisions depend on the 
similar decisions made by the opponent. 

The posing of the problem differs from those of the physical sciences in 
that the war setting is vague, complex, and unpredictable. No matter how 
we translate the situation into analytic terms we can never feel a comforting 
confidence that this is the “right” way to do it, that the answer we get will 
reliably concur with the actual outcome. 

The best possible procedure, it appears, is to coin and then analyze an 
assortment of models of the game. We should start with the simplest— 
mathematical formulations of the situation in which the essential question 
to be studied stands out bold and clear, and extraneous details are elimi- 
nated. But even here there will be many possibilities; there is latitude in 


won 






































310 THE APPLICATION TO WARFARE [11.4] 


choosing a payoff; there is more in constructing the kinematic equations. 
We then strive to incorporate more realism by adding new facets. One 
should try to adhere only to the most essential modifications, for with 
addenda to the problem, the analytic difficulties proliferate rapidly. 

With an assortment of solutions at hand, we should explore them for 
common features. It is quite possible that plausible modifications in the 
assumptions underlying a model can induce wide disparities in the 
optimal strategies. The Value, however, will generally be much more 
stable. Some study is now required to find what is consistent in the diversity 
of solutions. At times there will be remarkable features, such as a dictate 
that an optimal decision depends on the sign of an outré function of the 
state variables. When possible, we should probe such matters and try to 
understand them intuitively so that they can be assessed with some 
practical judgment. We should investigate the sensitivity of the payoff to 
the strategy, to our assumptions (the model), and to the values of certain 
coefficients appearing in the kinematic equations or in the formal payoff 
adopted. 

Our goal should be those aspects that are independent of the details of 
the model. If we find such, they will be valuable pieces of information. 
If not, we can probably draw conclusions such as, say, that it is not critical 
which of a certain set of strategies is used. Or possibly, say, that a certain 
type of strategy is optimal only in certain circumstances, but there it pays 
big dividends. Generally our conclusion should be that whatever feature 
is common to these models should in practice be an indicator for this type 
of strategy. 

This technique of a congeries of models accounts for our title phrase 
“broader problems.” The results will have to apply broadly if at all. It 
is seldom that we will have sharp enough data for a detailed particular 
situation with time enough to work out the solution. But, of course, until 
programs of the preceding type are actually carried out, we cannot tell 
what will come to light. 

Once results are attained, simulational gaming would be a means of 
testing them. We could pit a player (or team) using the results against one 
who is intelligent but ignorant of them and compare with plays of two 
knowing or two ignorant players. 

The drawback to this program is that at present there seems no possi- 
bility of its execution. The armed services sponsor a vast amount of 
abstract mathematical research, but they appear extremely hostile to any 
mathematical investigations of military matters, aside, of course, from the 
indispensable computations needed for equipment. The private firm 
cannot afford to, for the number of man-hours required is considerable. 
The pure mathematician is hardly likely to interest himself voluntarily in 
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such a program; if he has a bent toward applications he will prefer 
remunerative work. The foundations and other pecuniary sponsors of 
science, with their heavy financial obligations, must necessarily be chary of 
backing any scientific research (of any sort) that falls outside an established 
category or smacks of innovation. 


11.5. PROBLEMS OF FORMULATION 


We now examine in a general but more detailed way some of the 
constituents of a program such as proposed in the previous section. 


The payoff 

First, we wish to make clear an obvious but often misunderstood point.’ 
Whether in a game or a “programming” problem (one-player game), the 
term “optimal” has no meaning unless there is one quantity to be maxi- 
mized or minimized. Often there are several important factors of merit. 
But it is possible to optimize only one, and before a problem can be solved 
it is necessary to specify which one.” We can take into account several at 
once by employing as payoff a suitably weighted linear (or other functional) 
combination of them. But somehow the weights must be decided and this 
is almost always a matter of judgment. 

Now it sometimes happens that the solution is largely independent of 
the payoff. The strategy that captures most territory, for example, is 
likely to be close to the one that captures most materiel. There is no 
quandary then, but should the analytic results be sensitive to the payoff, 
and a weighted, mixed one have been used, then the results are no better 
than the judgment used in the weighting. A highly refined analysis, in such 
cases, is hardly worth while. 

Of course, there are many cases where there is no quandary at all. A 
defense system will seek to minimize the number of penetrating enemy 
weapons, for example, or warning system to maximize the probability of 
detection. But in complex battle games, what is the logical choice of a 
payoff? Frequently it will not depend on the immediate foray alone but 
will have to be integrated with at least a portion of the enveloping war. 
The subsequent utility of the outcome of the battle must be the coin of the 
payoff. 

To think of the logical ultimate, a bit of fantasy will help. Let us 
imagine a catalogue of military utility, which will list valuations of all 
sorts of military entities—weapons, munitions, personnel, bases. These 


* We have seen armed forces study contracts which outline a complex series of 
requirements and then ask for optimal action. 

7 Such is, of caurse, the universal dilemma of human judgment. When buying a car, 
we can seldom optimize price, mileage, speed, and appearance at once. 
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utilities will change with circumstances—supply at hand and potentially, 
special uses for special purposes, the enemy’s counter supplies. We must 
also have a set of rules for computing these changes. Then the payoff can 
be expressed in terms of total current utility of all assets that could be 
gained or lost by either side in the battle. 


The kinematic equations 

Setting them up also confronts us with alternatives. Once we have 
decided on the state and control variables, we must express the rates of 
change of the former in terms of both. It is at this point that we feel the 
imprecision of our subject most sharply. How are we to write definite 
relations between cause and effect of so indefinite a thing as a future 
military encounter ? 

There will be times when we can rely on data from experience or past 
analyses, but there will be times when we cannot. We will know that the 
greater the number w of weapons assigned against a number ¢ of targets, 
the greater the damage to the latter will be. For lack of better knowledge 
we often assume a linear relationship, 


į = —cw. 


Can we estimate the coefficient c, which means the (average) rate of 
destruction of targets by one weapon? Very likely, yes, although possibly 
coarsely. 

On such a basis we can construct kinematic equations, simply or 
elaborately. Most or all of them will be linear in the control variables. 
Then the essence of the technical problem is revealed by our theory. 
Of key importance are the singular surfaces—universal, transition, 
dispersal—associated with linear vectograms. Once they are discovered, 
the structure of the optimal strategies is at hand. 

To illustrate these ideas we now turn to the two typical problems already 
mentioned. As we have said already, a large amount of man-hours is 
needed to make practical headway, and we must be content here with the 
merest of beginnings. 


11.6. THE WAR OF ATTRITION AND ATTACK: 
A STUDY 


In the setting of a protracted war, each side must weigh the alternatives 
between direct combat and the raids on enemy sources of supply with 
possibly greater but future effects. Broad as such a problem is, we could 
think of it in still broader terms by endeavoring to admit the general 
problem of long- versus short-range tactics. 
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Example 5.4 seems to be the simplest version possible and a good one 
from which to start. The first kinematic equation is 


Ly = My — Cpe. (11.6.1) 


Recall that z, and 2, are the forces of the two sides, say amounts of aircraft, 
at time ¢; m, and m, are the unperturbed manufacturing rates; y and ¢ 
are the fractions of weapons devoted to attrition, that is, depleting the 
enemy’s supply. The foregoing equation states that the number of P’s 
weapons is subject to a decrease whose rate is proportional to yz, the 
number of weapons E devotes to this purpose at time £. 

Here is an instance of the linear assumption as mentioned in the last 
section. The meaning of c, in terms of, say, aircraft is the reduction of P’s 
rate of aircraft production due to an attack by one enemy plane. Is there 
any hope of rationally estimating c, (and, of course, ca)? 

Actually c, is the result of many quantities—the capability of the 
attacking craft, the nature and effectiveness of the defense, the type and 
strength of weapon (bomb), the vulnerability of bases and factories. If 
such components can be assessed, so can c,. But we must not forget that 
c, is complex only because the total picture is, and what we are working 
with is a preliminary simplification. 

Of course, there are many ways to advance toward reality—and com- 
plexity. One, which we shall actually probe later in this chapter, is that 
in which it is assumed that the weapon depletion is proportional not only 
to the enemy’s attack but also to the domestic supply. That is, the 
attrition destroys a certain fraction rather than a fixed amount of the 
target weapons. Thus the final term in (11.6.1) is replaced by cypa2. 

Another avenue is a three-way allocation, the new category being the 
assignment of certain weapons to the defense of “bases,” that is, they 
offset the losses inflicted by the enemy. To write the kinematic equations 
in such a case, we would first have to analyze the subconflict between these 
offensive and defensive craft. The result would be certain expected losses 
to both sides and an expected penetration leading to diminished pro- 
duction as before. But it might require an analysis longer than the current 
one. 

Of course, in any case we could take into account depletion of the 
weapons assigned to the various tasks. The simplest assumption, that of 
a fixed proportion, would adjoin to the expression for 4, terms like 


—I,pu, — LA — $)x, 
where /,, L; are the average fractions of weapons lost that are allocated to 


the two purposes. 
So far a homogeneity of weapons has been postulated. Reality would 
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require several types (bombers do not bomb only bombers). Some could 
be, as up to now, used for the attritional sorties, defense, attack, or any 
combination; different types would suffer different restrictions. The 
number of state variables is now enlarged, the problem is more cumber- 
some, but there seems no reason why, with the requisite labor, it could 
not be solved. 

Let us now turn to the payoff. It was supposed, in the early version, 
that there was a theatre of effective war detached from the attrition 
activity. In each time unit—for example, a day—each combatant assigns 
part of his weapons to this theatre, and they alone score in the payoff. 
The measure of merit for each side is the cumulative total—say the number 
of weapon-days—that he can deliver during a stipulated period of duration 
T. The payoff is the excess of these measures, 


fia- pa- 0- peat (11.6.2) 


so that each side strives to get a bigger weapon accumulation than his 
opponent. 

Another payoff would be simply the excess of extant weapon at some 
given future time T. It would be terminal with H = z, — 2. The 
desideratum here would be to store up the greatest possible excess in 
reserves of weapons over one’s opponent; the effective war is in the 
future, rather than concurrent with the attrition. 

One weakness is such payoffs is apparent at once; we have to know the 
duration Tof the war beforehand. With what reliability can T be estimated 
and what is the penalty of error? Should a revision of this estimate occur 
during the conflict, the format of our theory makes the consequent best 
alteration of the optimal strategies easy and natural.’ The consequences 
of a bad initial estimate will be softened thereby, and we should be able 
to assess how much by calculation. 

Still another type of payoff assigns different values to the weapons 
depending on the time of their entry into the main war. (An airplane 
delivered early gives more service than a late one, say.) Such amounts to 
inserting a factor, say F(t), into the integrand of (11.6.2). If F(t) approaches 
zero sufficiently rapidly with increasing ft, we can replace the upper limit 
by œ. Here then is a payoff independent of any prior estimate of the 
duration T. It expresses the surplus of the total aircraft value; the price 
is the obligation to choose the valuation function F. 

If we did not know T, but somehow felt we know a probability distri- 
bution for it, the payoff is of the form above with F(t) being the pro- 
bability density. 


® For T itself is a state variable, and the optimal ¢ and y are always functions of such. 
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The most rational payoff would be based on a detailed analysis of the 
main war, generally a much harder problem than the present one. If we 
could express the Value of this war in terms of the delivery of distributions 
of weapons by the two antagonists, this functional would be the logical 
payoff for the present game. 

But, among simpler payoffs, the imaginative reader will perceive many 
more possibilities, both of the genre we have described and others. The 
important point is not so much their realism or estimability, but whether 
the solutions—both the Value and the strategies—have traits essentially 
independent of the choice. 

Let us now turn to the analysis we have actually done: Examples 5.4 
and 11.9.1, the latter being placed later in this chapter to avoid inter- 
ruption. In the former example we find essentially simple strategies. Let 
us recall that c, is a measure of the destructive power of E’s weapons: 
one such weapon, when used for attrition, can cause a reduction of c, of 
P’s weapons per unit time. For P, cz is similar and we have supposed 
c1 > Cz. Then the optimal strategy for E, the player with the better 
weapons, is all-out attrition until a time 1/c, short of the scheduled end 
of the conflict followed by an abrupt switch to all-out attack. 

What does this mean? The 1/c, can be interpreted, in an average sense, 
as the time required for a certain number of E’s weapons to destroy a like 
number of P’s. Thus we can attach a certain reasonableness to the 
criterion. When there is a greater time remaining than the above, E has a 
favorable exchange rate: his weapons have sufficient time to eliminate 
more than their number of the enemy’s. As soon as this ceases to be so, 
it is advantageous to put all his forces into the direct war. 

But P, with an inferior weapon, makes a similar switch earlier.” He 
waits until a time which precedes the scheduled termination by 


L Jaai. 

Cy 
Is there a heuristic interpretation here? This result seems to transcend, 
at least superficial, intuition. Of course, we should expect greater com- 
plication. For P, when making his decision, must anticipate the optimal 
behaviour of his opponent, which entails a strategy switch during the 
time remaining. On the other hand, when E calculated his best time to 
change, he could count on the later persistence of P’s simple course of 
always fully attacking. 

Note that these strategies do not depend on the manufacturing rates 


° We are ignoring the curved part of 7, (Figure 5.4.2), where P switches earlier in 
anticipation of a weapon annihilation. 
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mı, Mma nor on the strength of the respective forces, x4, za, but only on the 
efficacy of weapons. Such, of course, is not true of the Value. 

By way of contrast, note the metamorphasis of the optimal strategies 
when we change the assumptions in the kinematic equations as we shall do 
in Example 11.9.1.1° Effectively the innovation is that c, now means the 
fractional, rather than the absolute, rate of depletion of P’s weapons by 
one of E. Similarly for co. 

We find that, with certain limitations, E makes his transition from 
attrition to attack at time 1/c,z, before the end. This criterion is rather 
like the one before and admits the same interpretation. But now the 
transitions no longer occur consecutively, for P utilizes, under different 
circumstances (see Section 11.9 for the details), the analogous T = 1/c22%». 

But more interesting are the universal surfaces. Again, under certain 
conditions, E (and analogously for P), before he reaches the strategy 
transition, should endeavor to hold P’s forces at the level 


a, = Vm,)c. (11.6.3) 


He plays all attack or all attrition depending on whether z, is below or 
above this value. Once z is there, Æ splits his efforts in the ratio 


y= 1a m/c, 


to maintain the state (11.6.3). He does so until T = 1 W m;cı, Whence he 
switches to all attack. 

What does all this mean? 

The next major step in a constructive program should be the study 
of the Value. How great are the penalties for deviations for the strategies 
just described ? 





11.7. THE BATTLE OF BUNKER HILL 


The general setting is of two approaching antagonists firing at each other 
meanwhile. Each being constrained in the amount of available ammuni- 
tion, the problem is to find the best distributions of their firepower, As the 
hit probability decreases with range, it increases during the approach. 
Firing too early then spells a probable ineffectiveness; firing too late 
allows the enemy too many unhindered shots. 

Again we have a situation redolent of basic or classic military theory. 
The particular realizations are, of course, numerous. We can think of 
nearing ships or naval task forces, as interceptor closing in on an armed 
bomber, or, as our historic title suggests, of two bodies of infantry. 


1° The reader should glance over this example now to make the following paragraphs 
intelligible. 
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Analyses of such problems are not new. Duel games, in the simplest of 
which the two duelists approach each other, with a single shot pistol, date 
from the beginnings of game theory. The ideas have expanded from here 
to multishot and silent duels," etc. and to cases of continuous fire, as the 
machine gun duel problem of John Danskin.” 

We shall study from our present standpoint and partially solve such 
games where the antagonists are corps which become partially decimated 


by the enemy’s fire. l , , 
One of the purest versions of such games has the kinematic equations: 


Èy = —2,Cop(T)y 
dy = =x p(T) 


my, = —cy 
Mg = — CW 
t=-1, 0<dy<i. 


Here x, and x, are the numbers of soldiers in two approaching armies, 
which are firing on each other meanwhile. The time until they will meet 
being T, the hit probabilities, p, and pz, are decreasing functions of T. 
The quantities m, are the (average) ammunition per man in the two armies 
and the c, are maximal possible firing rates. Each side has the option of 
firing at any rate up to these; taking the fractions as the control variables 
gives us the third and fourth KE. , 

The total number of bullets fired per unit time by Army 2 (that is Æ, 
the minimizing player) is thus x,cyy. On the average we will suppose that 
the fraction p,(7) of these hit their mark; the total which do is the rate of 
decimation of Army 1. Such accounts for the first KE, the second 
following symmetrically. 

Observe the precise meaning of p,. Its reciprocal is the average number 
of bullets needed to kill one enemy, a quantity which we suppose to 
decrease as the armies get closer. 

The game terminates when the armies are close enough for the above 
approach phase to be no longer a valid description. With a suitable time 


11 See the account in Reference [10]. 


12 Reference [11]. cn 
18 The total bullets in Army i is s;m; and the rate of change of this is tn, + £t. 


The first term is the effect on the firing rate due to the firers being decimated. We have 
neglected it, on the grounds that in a reasonable battle it should be small compared to 


the second term, the rate due to weapon use. ‘ f 
To take this effect into account is not hard. We form differential relations, as the above 


KE, and solve them algebraically for the ż;, m, to get our final KE. The results is more 
cumbersome in form than what we are using. 
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scale such occurs when T = 0, which is taken as defining @. Of course, & 
is the set where all five state variables are nonnegative. 

The best firing schedules (strategies) depend on what is chosen as payoff. 
Because the decision is typical of the quandaries facing the military analyst, 
let us look further at this instance. 

One (cold-blooded) choice might be the difference in the number of 
surviving men. Such implies that we take 


H = t, — z (11.7.1) 


But note that our solution will reflect the choice. We should not be 
surprised if one or both sides arrive at the fray (after the approach) 
devoid of ammunition. For (11.7.1) puts the objective in terms of sur- 
viving men only; then, to optimize, each side will clearly want to use all 
its ammunition, if there is enough time (T) for it to do so. 

To remedy the unreality of such an analysis, we might use as payoff the 
surviving excess firepower. That is, we take 


H = mx, — mt. (11.7.2) 


If the surplus of men and the usable ammunition are both important, 
we could use for H a weighted linear combination of (11.7.1) and (11.7.2). 
But the weights (coefficients) are a result of a judgment, and we cannot 
expect the solution to be better than its validity. 

Again, it might be that Army 1 is defending such a vitally important 
target from the invading Army 2, that the men of the former are expend- 
able in comparison. Then both sides would be interested in the men or 
firepower that breached the defenses. Thus a suitable H would. be z, or 
Mig. 

But there is only one strictly logical way to choose a payoff. We should 
consider the encounter which arises after the approach of the two armies 
and analyze it first as a separate problem. Its Value can be expected to 
depend on the x; and m; which were its inputs. Then this function of these 
four arguments should be the H of our original game. 

Suppose now that H depended only on the z, such as (11.7.1). Observe 
that if the third and fourth KE were deleted, we would have a self-con- 
tained differential game with the state variables: Zi, X, T. Its solution 
would entail 6 = # = 1, for clearly the players do best by full firing at all 
times. But then our purpose, which is to find the best usage of a limited 
amount of ammunition, is lost. The role of the above two KE then is to 
supply these conservation constraints. 

We shall partially solve two cases with payoffs (11.7.1) and (11.7.2). 
Both appear to have elaborately variegated solutions, and to avoid a 
gallimaufry of special cases we shall try to simplify. In both, we shall ignore 
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the bounds on the z,. This allowing of negative numbers of men is not as 
absurd as it sounds. First, it is likely that in the cases of realistic interest, 
annihilation of neither side will occur; in fact, reasonable ranges of the 
variables will likely bound us well away from such a possibility. Secondly, 
the solutions obtained will be parts of the full solution. To extend them 
we need but add new components to on which a requisite variable is 
zero. Let us, say, adjoin %,, defined by z, = 0 (and also z, > 0; other 
state variables > 0). The subsolution in @, is trivial: the extinct Army 1 
(P) cannot fire and E certainly will not; then $ = ~ = 0, and to find V 
on Ï; is trivial. We use it as an H with which to construct retrograde 
optimal paths emanating from @, back into €. The solution is completed 
by merging paths of this type with those of the unconstrained solution. 

When we use the firepower payoff (11.7.2), we shall also ignore the 
Positivity of the m,. It has already been explained that such would be 
absurd for H = x, — a, but it is permissable in the present case on 
grounds like those of the two preceding paragraphs. It is not likely that a 
side will exhaust its bullets to attain a good payoff expressed in terms of 
them. If we wish to study such cases we can adjoin auxilliary @, as above, 

For reference, we write out the ME and RPE relative to the foregoing 
KE. 

A$ + A: — Vp = 0 (ME,) 


where 
Ay = =(P: Va + Vs) 
Ay = —C(%ap2V, + Va) 
and 
a hee .  flifA,>0 
—loifa4s>07 ”7 ree <0. 
while the RPE are 
Tı = tp) Vi = — cP: hh 
f= 20,6 h= — CPV Y 
m = c$ Ve =0 
Me = Cop V5 =0 
io 1 Vn = — c% Pi Va — Ca£a Po Vi. 


Example 11.7.1 The Battle of Bunker Hill: firepower payoff. We take as g 


T =S, M= Ss T=0 


: Ta = So, Mg = Sy, 
and, from (11.7.2), 
H = 554 — S183- 
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Completing the initial conditions, as usual, by 


vy =E 
Os; 
we have, on @, 
Vi = —Ss, V; = =s, 
Vz = Sy, Vy = Sy 


and so, still on Q, 
A, = —¢,5,[54p(0) — 1] 


Ag = CeSq[—S3p2(0) + 1). 


There is full or no fire at termination depending on the signs of the 
brackets. Thus P, say, near termination, fires fully (6 = 1) when s,(= m,) 
> 1/p,(0) and not at all in the reverse case. When Ẹ = 1, the test of 
Section 7.10 shows a void and there will be a universal surface.14 We have 
not calculated it—a proper five-dimensional affair—but shall reflect on its 
significance. © 

It consists of those states at which P should exert such a partial rate of 
fire so that at termination m, = 1/p,. This means that the enemy (£) 
soldiers each have (on the average) just the number of bullets which would 
give each P-soldier the capacity to kill (on the average) just one man. 

An odd criterion? We should recall that this model depicts a sort of 
contest in the economy of bullets; each side expends them such as to have 
the maximal surplus in the end. The theory has furnished a criterion for 
doing so which does not appear obvious. (Of course, we recall that it 
embraces more states than just those of the universal surface; one side 
full fire, on the other none, until the US is reached.) Does it have a direct 
interpretation? 

But the above type of action probably lies outside the range of practical 
interest. The one-man kill capacity implies that the approach phase was 
so long that it paid E to expend nearly all his ammunition during it; 
such would be expected to hold only in rather extreme cases. 

The more realistic states, we should judge, will lie well on $ = 1 side of 
the universal surface. Then the partie concludes with full fire. Were this 
preceded by a no-fire phase, we should have a corroboration of Warren’s 
command. That it is actually the case, under reasonable circumstances, will 
appear from Lemma 11.7.2, later in this section. For simpler analysis, we 
have assumed p, = pz, and the remark that will follow the proof shows 
that the conclusion is a likely one in practice. 

We have not computed the transition surface shown to exist. But, at 
least, when p, = pz, this could be easily done with the aid of Lemma 11.7.1, 


* It is not hard to show that if # = 0, no ¢ — US can exist. 
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the preceding A,, and the initial conditions. The surface would consist of 
those states when the “whites of their eyes” are visible. 

Finally, it is likely that the full solution will embody a universal mani- 
fold for both players, that is, one to which both will steer. On it both 
contro] variables will assume intermediate values, say $ and %, and it can 
be expected to be of lower dimension (< 4) than a surface. 

Because we have developed no general theory for such manifolds and 
have no grounds but instinct for asserting the existence of one in this game, 
the following must be regarded as spectulative. 

Along a path on such a manifold, the intermediate control variables 
require that everywhere A, = A, = 0; then from the main equation, also, 
Vr =0. The time derivatives of these three quantities also vanish. 
From A, = 0, we have, using the RPE, 


CaP P lE Va — V1) — pi (EV) = 0 (11.7.3) 
and also Vr = 0 gives, directly from the RPE, for some å, 
Vi = —Aeyx,py 
Va = Aceiap,'. 
Substituting these into (11.7.3) and solving gives us 
Cato Pi" Da! 


CaP p(c12 p: = Cata pr) l 





y= 


The analogous expression for ¢ is obtained by exchanging the indices. 

Writing ¢ and 7 into the (left) RPE, we obtain a system of differential 
equations in the state variables. With a proper set of initial values, their 
integrals may give us our universal manifold. 

Of course, final and more comprehensive statements require a more 
extensive analysis than we have undertaken here. 


Example 11.7.2. The Battle of Bunker Hill: manpower payoff. The version 
with payoff being the excess of manpower at termination (H = 2, — 2,) 
would apply to cases in which the events succeeding our battle of approach 
do not depend on the munition supplies. 

Our treatment again shall be partial. We shall assume one side has 
enough bullets for continual fire, that is, @ = 1 always. Thus our problem 
will concern only the best distribution of the opponent’s limited fire and 
so will be a one-player game. 

This is not as drastic a limitation as it at first sounds. First, our 
limited problem is a good one in itself: how, for example, best distribute 
a small fusillade from heavy weapons against an approaching enemy who 
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maintains a light but steady barrage at all times, the accuracy of fire in- 
creasing with time for both sides? Secondly, its solution covers most of the 
essential facets of the complete case. If the opponent always does not fire 
(p = 0), the solution is trivial. Thus both steady possibilities of the 
opponent are covered. If the roles of the players are reversed, then the 
only features of importance omitted are possible universal manifolds of 
small dimension, where both players use intermediate values of their 
control variables and simultaneous transitions of them. 

Our problem now has one control variable, ¢. The state variable, m,, 
can be suppressed; it plays no role. The KE are obtained from the 
previous set by placing y = 1 and dropping the fourth one: 


ty = —2XCop(T) 
ty = —2%c,p,(T)¢ 
t, = my = —C, 
T=-1. 
The ME is 
Ad — CtP Vi — Vp = 0 
1(A < 0) 
O(A > 0). 
The RPE are the same as before if we drop the fourth entry in each 
column and replace # by 1. 
The universal surface for these KE has been computed in Example 


7.9.2. 
We will also need the readily computable result 


A = —c,M 
where M = copyp(%2Vq — £V) + t Vapi. 


where A(= old 4,) = —c,(%,p,V¥2+V,) and ¢= 


(11.7.4) 


If P has enough ammunition to last until termination, his optimal 
strategy is obviously steady full fire: ¢ = 1. We are led to something like 
Example 7.14.1, in which the surface bounding such cases (see Figure 
7.14.1) will probably act as a semiuniversal surface. Therefore we will use 
this surface as a terminal one and call it @,. It is characterized by there 
being just enough ammunition to last until T = 0; we parametrize it: 
6: t= s 

Ta = So 
z = m = cs % 
T=s. 


15 Because of its prominent role, we write s for ss. 
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There is another significant terminal surface. It may be in some cases 
optimal for P to do all his firing early and exhaust m, prior to termination. 
We encompass such a possibility by the terminal surface 


Gz: Ti = Sy 
Ta = So 
m=0 
T=s. 


We shall treat @, first. To ascertain H on it, we see how play will pro- 
ceed from a typical one of its points. We obviously construe m, = 0 as 
implying a mandatory ¢ = 0, and 2 and z, will satisfy 

Éi = XCop(s — t) 
t, = 0 


(11.7.5) 


and x, = s; when t = 0. The s — t appearing is T when it starts from the 
same point, (s,, S2, 5) of Ga. The sought H will be the value, as above, of 
ta — x, when t = s. The above system integrates to 
t 
y= 5) — casa | p du 
= 


Tz = S2 
and so 


H = s(1 + af p(u) du) — S. 
0 
Our usual method then gives on 6, 
V, = 0H/ds, = —1 
Va=i+ af P2 
0 
Vr = S2Csp,(s). 
Putting the initial conditions in the ME, it becomes 
Ag = 0. 


If there are to be paths entering @,, they must do so with ¢ > 0 and 
so A = 0 at such points, and consequently 


V= -sp1 + af Pa| <0. (11.7.6) 
0 


These tributary paths into @, can exist, as A = 0 there, only if A <0 
or if 
M = M(s, Si, 52) > 0. 
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M<0 


k—m>o 


mi 


(d) 


(c) 


[11.7] 





M>0 


Figure 11.7.1 
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From (11.7.4), we find that 


M = cop,(s)p2(s) [s(t + Cy f | + s] +s (: + a'r) pi (s). 


After evaluating this function on &,, we will have tributary paths when 
M > 0, that is, such will be states where P optimally concludes firing, and 
we should expect singular surfaces to emanate from the boundary where 
M=0. 


When s = 0 
M = eap (Ops? — z] (11.7.7) 
1 
where T= ZPO 


C2p:(0)p,(0) 


and the solution depends rather critically on the sign of J (recall that 
Pi (0) < 0, as the p, are decreasing). 

If J < 0, then, for small s (near termination), M > 0 and there will be 
cases where P should do all his firing early. If J > 0 such tributary path 
play occurs only when the force ratio s,/s, is larger than J. Thus for a 
certain subset of cases (near @, with s small and J > 0) P should fire early 
when his force is smaller than that of E and late when it is larger. To have 
J > 0 roughly means that P’s firing accuracy falls off rapidly with range 
when the forces are close and cz, E’s rate of fire, is small. (We neglect the 
effect of p,(0) which should not be too far from 1.) 

Due to the four-dimensional &, we shall restrict our graphical depiction 
to the (7m, T) plane. In such a diagram, the paths with ¢ = 0, where m 
does not diminish, are horizontal, whereas if $=!, they have the fixed 
slope, —c,. In Figure 11.7.1a, @, (rather, a section of it) appears as a line 
of the latter slope through the origin; to the left is the region of surplus 
m, where 6 = 1 always. A similar rendition of @, lies on the T-axis, 
where m, = 0. 

There should be a distinct such chart for each value of (Si, S2), but they 
are not cross sections of & for a fixed (2,, x.) because the latter changes 
value during a partie. Nevertheless, each play of the game will be ac- 
companied by a moving point on such a chart which ultimately reaches 
€, or Ca 

At (a) and (b), M > 0 for small s (= T when m = 0). If for some 
s (= T,), M changes sign, the diagrams suggest that the strategy switch 
might entail a transition or universal surface. Similarly, if M < 0 at 
small s, (c) and (d) indicate that a sign change generates a transition or 
dispersal surface. Thus it becomes important to study M as a function 
of s. 
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We shall not do so here meticulously. We shall suppose 


Pi = Pa = P(t) = ae™ 


with 0<a<1,k> 0. No versimilitude is claimed for such probabil- 
ities; they merely fulfill our requirements (p; < 1 and decreasing), yet 
lead to simple calculations. 

Then 


and 
M= —r|(®) (can = [a(t + at) + 2s, (eap + sk(1 + s), 


The roots on the right are at p = 0 (s = œ) and 


So(1 + cajk) + 2s, + V(S2 + czxajk)} + 2s)? 


RF: 2seCo/k 





Clearly, the latter two are positive. To correspond to a positive s, root < a. 
It is not hard to show that such cannot happen for the larger root, and for 
the smaller it happens if and only if 


Thus, if either 


or —~>J>0 


M > Ofor small s, but will change sign; one of (a) and (b) of Figure 11.7.1 
will occur. To know which requires still further analysis, which will not 
be set down here. The criterion is the sign of M at when t = 0, s = T; 
and such involves c,, which has not appeared in the preceding criterion. 
Our own incomplete evaluation seems to indicate that either the transition 
or universal surface could appear. 

In the event of the latter, if Lemma 11.7.2 applies, there must still be a 
transition surface for the $ = 1 paths; such as indicated by the broken 
curve at (b). 

But if s,/s, < J, M < 0 for all s. Then (c) and (d) cannot occur; the 
horizontal paths persevere rightward of @,. Here is pristine validity of 
Warren’s command; P witholds fire completely until the last instant that 
permits its exhaustion. 
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The analysis for @, parallels that for Ga In place of (11.7.5) we use 


% = —X2Cop,(s — t) 
t = =t p (s — t) 


but from here on the procedure is the same. There is a new M, but it 


agrees with old when s = 0. Thus the criterion near the origin is just as it 
was. 


, ban ee grounds for plausibly conjecturing (our analysis is too 

incomplete for certainty) that the conclusion of t 

aaea, wo paragraphs back 
Warren’s command is valid if 


Za <O 1. 
zı — Cap,(0)p,(0) 

But, of course, our partial analysis indicates that the full solution will be 
quite complex. For instance, if there is a universal surface bordering on 
Ga, it would entail firing programs of consecutive phases such as, first, a 
period of no or full fire (tributaries), second, partial fire (on the US in 
accord with (7.9.19), and third, full fire (on €). 

We conclude with the two lemmas cited several times. 

When p, = p (= p) the RPE can be integrated in closed form under the 
full fire case when 6 = Ẹ = 1. The result is immediately extendable to the 
case where p(T) and p(T) have a constant ratio, but when they are 
essentially distinct, we are led to a form of Ricatti’s equation. 


LEMMA 11.7.1. The integrals of the system 
y = Catap(7) 
Sa = ¢,%p(7) 
with initial conditions: x, = s; are 
2 =Q + a Q7 


t, = 8,0 + 8-0- (11.7.8) 


where 


0 = O60) = exp (Yaa new du) 


and 


1 
x+ = 3 (sı + Sex/C9/c;) 


Pa = 5 et wale). 
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If the V,, with the same RPE, have the values S, when J = 0, they are 
also of the forms (11.7.8) but with 


a TO F Sala) 
jua : (Sa F S/a). 


The proof is, of course, direct. 
Suppose at some late stage in the game, $=9 = 1. We want to know 
if there are transition surfaces so that earlier ġ or Ẹ was 0. 


i i i = Pa (= p), there is 

LEMMA 11.7.2. If, in the Bunker Hill game with p, = pa (=p 
a set of points S at which $ = Ẹ = 1 and V, < 0, then at some earlier 
time the paths through S will have met a transition surface preceding which 


$ = 0, provided that either 
(1) cr p(T) dr exists or 
0 


a 
~ T= 
(2) p(T) T’ 00 


and 2a fcc — 1 < 0. (11.7.9) 
Proof. The value of $ depends on the sign of A,, which by hypothesis 

must be negative at S. We are to show that it becomes positive for large 7. 
From Lemma 11.7.1 


A, = —¢,(p(r)[kQ? + OD] + S) (7+ œ) 


for some constants k and S3, the latter being the value of V; at a point 
the KE informing us that V, is constant along a p The form of the 
i ’s bei i i tion of 7. 
bracket is due to Q’s being an increasing func c 
As S; < 0, if the term preceding it becomes small enough for large 7, 
our result is attained. Such happens under (1), for Q becomes Saad 
and p, zero. Under (2) we are interested in, where k’ and k” are constants, 


2 exp 2,/¢1€a(k’ + a log 7) 
T aa 
— k'y cica —1 


so that (2) implies our result. 


Let us note that, in three-dimensional physical space, the hit prob- 
abilities diminish, for large ranges, inversely as the square of the range. 





[11.8] SOME PITFALLS IN ADAPTING GAME THEORY TO WARFARE 329 


If the range in our game decreases lineally with time, we will have 


1 
ro = ofz) 
so that (1) will hold. 

In the less likely case of two-dimensional aiming (2) will apply, and the 
validity of Warren’s command would depend on the truth or falsity of 
(11.7.9). 

Let us further note that the hypothesis S, = V, = aV/Om, < 0 can be 
counted on. Because Army 1 is minimizing, we should expect the Value to 
be smaller the more ammunition he has. 


11.8. SOME PITFALLS IN ADAPTING 
GAME THEORY TO WARFARE 


In the application of a mathematical theory to a practical science there 
are certain unavoidable limitations, Such general derelictions hold when we 
utilize game theory as a military tool as well as some peculiar to our subject. 
We list 


1. An optimal strategy is, by its nature, the best possible, but it may be 
complex and the practical gains over a simpler, more obvious competitor 
slight. Such is a common phenomenon in many reaches of applied mathe- 
matics, but in game theory there is the innovation of an intelligent (at 
least so postulated) opponent. We must ascertain whether he can seriously 
exploit a deviation from an optimal strategy. 

2. There are instances when we have assumed knowledge on the part of 
a player of certain parameters which he might not have in reality. (Note 
“parameters,” a different affair from not knowing the opponent’s state 
variables. The latter situation occurs Properly in games of imperfect 
information and will be discussed in the next chapter.) 

3. Let us recall the von Neumann definition of the solution of a zero- 
sum, two-player game. When both players utilize optimal Strategies, the 
payoff will be the Value. But if one player plays nonoptimally, there will 
exist a strategy of his opponent enabling the latter to do better than the 
Value. 

There are cases when the only such existing counterstrategies are non- 
optimal for the opponent. Then what is he todo? To exploit the defection 
of the first player, the Opponent must play nonoptimally himself and so 
expose himself to a similar risk, 

In differential games, fortunately, this phenomenon generally does not 
occur. If one combatant acts optimally and the other does not, the latter 
automatically suffers a penalty. 
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11.9. WAR OF ATTRITION AND ATTACK: 
SECOND VERSION" 


In Section 5.4 we investigated a war game in which the control variables 
¢ and y were the fraction of his weapons that a participant devoted to the 
destruction of his enemy’s weapon production. His remaining weapons 
were devoted to the theater of war proper, and it is these alone that 
contribute to a favorable payoff. 

The solution showed that if, say, P acted optimally, $ was at first 1 (all 
weapons used for attrition) and then at some definite later time he switches 
to $ = 0 (attack with all weapons). No intermediate weapon allocations 
were ever employed. They are in the version below, revised slightly 
toward more complexity and better realism. As is generally true with 
linear vectograms, the agency is universal surfaces. 

Our new KE will be 

Ži = My — CPT Ta 
ig = Ma — Catta, 0<dy<l 
T=4 =-—1 
with, as before, 
G = (1 — y)t, — (1 — om. 

As in the earlier version, ¢ and y are the proportions of their forces that 
the combatants devote to the long-range purpose of destroying the enemy’s 
forces at their bases. The residual fractions, 1 — ¢ and 1 — y, enter the 
battle proper and contribute directly to the payoff. When ¢ or y = 1, we 


will thus speak of attrition; when it equals zero, of attack. The innovation 
here lies in the final terms of the first two KE and was discussed in 


Section 11.6. 
As in the earlier version, & is 
120, %20, 420 
and @ is 
%=5 20, %=5,20, 2 =0. 
Again it seems inadvisable, at least at first, not to bother with the 


maintenance of 2, % > 0. In fact, we shall not push the solution to 
exhaustion but shall be content with exhibiting its novel and important 


aspects. 
The ME, is 


mV, + mV, — Va + £z — 21 — %4S,h + 28,9 = 0 


16 Likewise posed by A. Mengel. 
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where Si = CVt — 1, Sp = =c Viz — 1, 
Thus, clearly, 
=Í if APN, P=fi if S,>0 
0 if S&<0 0 if S,<0° 


Turning now to the initial conditions, on € we have Vi = V; = 0 so 
ae a Sa = —1 and, as we should expect, $ = % = 0, that is, all 
attack and no attrition late in the war. Near ¢ <r pie 
ep r @, the RPE are (V, proves 


o fe] 

4 = —M;, V,=—-1 

o o 

Tı = —M,, V: =1 

o 

T=1 

which have the integrals 

zı = Sı — my, V = =r 
To = Sg — MT, Vy = 7. 
T=r. 


Thus 
Ss, = CaT(S — mor) —1= CoTt, — 1 


S: = ¢7(s, — mr) -1 = (Tx, — 1, (11.9.1) 


Thus transition surfaces occur on the h i i 
yperbolic cylinders obtained b 
equating the final terms of (11.9.1) to 0. Such surfaces, denoted by 7. y 
and 7, have the appearance of Figure 11.9.1. The analysis thus far ia 





Figure 11.9.1 
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Figure 11.9.2 


valid at most.in the region behind their configuration as sketched (that the 
subset of & for which 
1 
Tx, < 1 and Tx, < 1), 

C2 ci 
Not all of this configuration will be significant. For let us envisage é, 
for example, as projected on the (xa, T)-plane (that is, aview of Figure 1 1.9.1 
from along z,-axis). The paths will appear as the oblique straight lines of 
Figure 11.9.2, and 7, as an hyperbola. Let it be tangent to the paths at the 
point Q. (Q is really the endwise projection of a line Q in &.) Clearly 
only the part of 7, above Q will act as a transition surface because below 
Q, 7, will be met by none of the (retrograde) straight paths emanating 


from @. 
A brief calculation shows that Q has the coordinates 


T = 1) mca 


i i i leads to a change 
We can expect (and in fact can verify) that crossing 7 , 
in ¢ from 0 to 1, and so on the right side of 7, one of the RPE will be 


replaced by 


(11.9.2) 


£a = V Mifc, 


Ta = —M, + Catta. 

Except at the boundary (x, = 0) the final term will be positive, and the 
paths emanating on the right of 7, will have greater slope than our former 
straight paths. Thus there will be a void just to the right of Q, and we act 
on this cue to seek a possible ¢-US. 

As on the straight paths, #7 = 0, the universal surface will be sought on 
this premise. (Further, our criterion for a ¢-US shows that none can exist 
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when y = 1.) The analysis was done in Example 7.9.3, and we found 
there the very appropriate candidate (7.9.23): 


T = Vmes 


Such is a plane through Q; of course, we utilize only the subset lying to 
the right of Q where 


T > 1/Vmc,. 
To navigate on this plane let ¢ be ¢. Then from the second of the KE 


Ža = 0 = m, — cdz / maca 
or $ = (1/2) / m. 


As our constraints require ¢ < 1, the universal surface is confined to 


(11.9.3) 


a, > Vma (11.9.4) 


Our thinking will be simplified if we temporarily convert the situation to 
a one-player game by taking y = 0 always. In particular, we are disen- 
cumbered of 7. We are then dealing with a situation in which E attacks 
directly only, that is, he does not endeavor to destroy P’s resources. 

The universal surface appears as the quarter-plane BAC sketched in 
Figure 11.9.3. The tributary paths, as we already know generally, will 
merge smoothly with the family ($ = 1) emanating from 7, and with the 
family (¢ = 0) coming directly from @. The paths on the surface itself are 
sketched in the figure. 

Let us interpret matters from out present state of knowledge. If ter- 
mination is still far off (T large), and his forces not too small ((11.9.4) holds), 
P should attack ($ = 0) if his enemy’s forces are below the level V Mal Cy and 
do so until they rise to this level. But if they are above, his strategy 
should be attrition until they are reduced to y. m,/co. When z, = y MoCo, 
P holds it there by mixing attrition and attack in the proportion (11.9.3). 
He perseveres in this policy until the time 1 / mac, prior to termination, 
when he switches to full attack. 

But against a large enemy force and with not too great a time to go, P’s 
attrition policy will attain 7,, and he switches to attack when this time 
T = 1/ep2.. 

These ideas are again illustrated by the cross-sectional diagram of 
Figure 11.9.4, 

What happens near the hiatus bounded by QAC in the preceding figure? 
We shall show that there is a dispersal surface meeting it. 















































334 THE APPLICATION TO WARFARE [11.9] 





xı 
Figure 11.9.3 


Relooking at the KE with $ = 1 (and ¢ = 0): 

EA = -m (1) 
= —M + Ctt (2) 
t= 1 (3) 


let us construct initial conditions on the universal surface near AC. For an 


R 
© 
I 





Figure 11.9.4 
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initial point on AC itself (2) shows that for r = 0 

ty = 4/ Maf Cz, ča = =m; + ca(/ma]c2) majca = 0 

Ta = Geis + Tita) = — cam / mafea <0 
so that the paths, as they leave AC descend like 72. From starting points 
near AC, the paths will at first rise (¢, = —m, + eV. Mafca + €) V Mfc = 
small and positive), then, by continuity, descend at this kind of rate. 
(One is sketched in Figure 11.9.3.) Thus they will meet the straight paths 
emanating from the lower side of the universal surface. From 7 1 near 
AQ, as z is still smaller, the paths will descend even faster, Thus a dispersal 
surface can be constructed (Section 6.5) by equating the V on the two sets of 
intersecting paths. 

Now let us restore the two-person game. First it will be convenient to 

know a little about V on the universal surface. We describe the latter by 
fresh parameters: 


T1 = 5, > VMC % = Vmaco, T=s > 1/V macy. 


Then we can show fairly routinely that on it 





Vi = —Ss3, Vz = 1/ V mcs, Vr = 24//ma/ca ~s mMıS3 


and also 
S = — c Vir — 1 = css — 1 


and the surface is only acceptable in the two-person game at points where 
Sa < 0 as its construction was premised on p = 0. This means that we 
must discard all of it except the portion which lies behind 7, which has 
the equation c,t,T = 1. The boundary is indicated by the dashed curve 
in Figure 11.9.3. 

We have no assurance that 7, aside from this dashed curve, will play 
its former role. What we must do. is to evaluate S, along the tributary 
paths from the residue of the universal surface and ascertain the set of 
points on which it vanishes. Doing so for the paths below the surface is not 
difficult; we find the old 7, still in business. But the paths from above 
lead to differential equations recalcitrant to elementary methods, and we 
have not pushed matters to completion. Our conjecture is that Sz becomes 
zero but on a surface not 7. 
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CHAPTER 12 


Toward a Theory with 
Incomplete Information 


12.1. INTRODUCTION 


The value of extending our theory to games of incomplete information, 
for military and other applications, is clear. But the achievement is 
difficult and until recently little progress has been made. This chapter 
largely sows seeds rather than reaps results. It portrays the ete 
conjectures what solutions would be like and seeks to delineate the mos 

ising ways of attaining them. 

Pik the following section is the first glimpse of the inevitable S e 
of mixed strategies. As in the standard theory of the zero-sum, oop 
game, such consist of a player’s decisions being made in accord wit! 2 
definite probability distribution. Just how this should be donein AO 
games is not obvious. The most likely possibilities are suggested : y 
realistic prototypes of the games. After a tentative general definition ofa 
differential game with incomplete information, we turn to specific m 
using as models some actual instances of firing and pursuit games. The 
search game, as one with minimal information is termed, often appears 
likely to furnish a constituent of the solution of more general cases. 

Search games constitute Sections 12.3 and 12.4. , In the first, we prove 
that when the hidden objects are numerous and immobile, the time to 
find them (the payoff) is nearly independent of the searcher’s ee 
long as no effort is wasted researching territory already scouted. : K 
striking result, as far as we know, is the only one definitely establishing : 
practical validity of approximate methods, an approach we feel has grea 

possibilities. This cause is argued in some detail in Section 12.6. 
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Section 12.4, on search games with mobile hiders, again takes us to an 
unexplored realm. We conjecture here that the details of the randomiza- 
tion are unimportant, but certain basic parameters, such as the hider’s 
speed, are not. Again there appear to be grounds for an approximate 
theory. 

For certain games with a “stationary” or “‘steady-state” character there 
appears to exist a technique for solution, albeit a tedious one (which might 
be alleviated by approximation). It is expounded in Section 12.6 and 
exemplified by a firing and evasion game. 


12.2. A SPECULATIVE PURVIEW 


One of the main difficulties of differential games with incomplete in- 
formation is that undoubtedly optimal play will, in all essential cases, 
require mixed strategies. What form will they assume in a differential 
game? A mixed strategy customarily means the randomizing of a player’s 
decisions in accordance with some probabilistic law. Just how is a player 
to randomize his choice of control variables, quantities which he controls 
continuously ? 

Whatever the nature of the strategies, there is a basic distinction from 
games with full information. Generally strategies can no longer rest on the 
concept of the control variables being functions of the state variables, 
that is, a player’s decision depending on the current position only. With 
partial information his knowledge of the past states will generally motivate 
his current decision. 

The form of the information enjoyed by a player will be a probability 
distribution over the state variables at each instant. As full knowledge is a 
particular such distribution, the games with complete information, 
discussed heretofore, are special cases. 

There are two sources of this information. Some is granted, that is, 
furnished? by rules of the game which may be formulated for this purpose. 
For example, if the game were modeled on a physical situation in which 
one player obtained knowledge of his rival through an imperfect detection 
device, the rules might be designed to simulate the imperfection by supplying 
him with equivalent granted information. Each player must also be 

granted some data as to the starting position; the most general form is a 
probability distribution over £. 

There is also inferred information. A player will know the full history of 
his own control variables? and he can infer from it more than the granted 
information. 

1 The case where no information is furnished is not excluded. 


* Such is usually the case, but there are exceptions. For example, if one “player” 
is a team whose members are in imperfect communication. 
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Example 12.2.1. Let the kinematic equation be 

G=2+¢ 

y=2+y, —-l<d¢y<l. 


Here P, controlling ¢, will always know the current value of x if he knew 
its starting value x° (by integration of the first KE); such is inferred 
information. If he had only a probability distribution of 2°, at any later 
time his knowledge would be this same distribution translated by a fixed 
amount, which is always known to him via ¢(t) through integration. 

But if he knew yp, the starting y, all he could tell of y at time ¢ would be 


Ytt<y [SY + 31. 


But there may be further rules of the game that supply him with a finer 
knowledge; such would be granted information. 


Of course, it is quite possible to have optimal pure strategies in cases of 
only partial information. But in many of the more significant examples, 
based to some extent on reality, it is intuitively clear that mixed strategies 
are demanded. 


Research Problem 12.2.1, Let us suppose a game of incomplete infor- 
mation with one control variable apiece for the players and linear vecto- 
grams. In the solution of the corresponding game with full information, ¢ 
and # are to assume their extreme values almost everywhere in £; let us 
say & , are the subsets of £, where $ = +1. Then, returning to a partie of 
the original game, at some instant P will have available a probability 
distribution, granted, inferred, or composite, for x over @; he can then 
always compute the probability that x is in &, or &-. A plausible strategy 
is to play, according to which is greater, 6 = +1 or —1. Under what 
conditions is this strategy optimal? (One might use Example 12.2.1, with 
various @, G and H to experiment along these lines.) 


For less abstract examples, let us consider pursuit games. As usual P 
pursues E, E evades P, but each (or possibly only one) has incomplete 
knowledge of the other’s position. Reality, especially the limitations of 
actual sensing devices, suggests a number of forms for this incompleteness 
(for definiteness we shall speak to P’s granted knowledge of E, but of 
course all possibilities may hold reciprocally): 


1. P may know £’s location (rather) well but may have little information 
as to his other state variables (if there are any), such as heading and speed. 

2. P’s sensing device gives him only a probability distribution of E’s 
position. 

3. Pmay know E£’s relative bearing only, that is, the inclination of line PE. 
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4. There may be a time lag; P can act only on E’s position some fixed 
time T ago. 

5. P receives his information intermittently. (Suggested, of course, by 

sweeping radar, but there is game-theoretical interest only if the interim 


between signals is great enough to allow E to maneuver significantly 
therein.) 


When no information? is granted to either player, we will speak of a 
search game. 

We will discuss some possibilities indicated by some of the five categories.* 
Let us take a simple cookie-cutter version of 2, that is, at each instant the 
only information granted P as to E’s whereabouts’ is that Eis equiprobably 
within a sphere of radius r. In a discrete model, where the players alternate 
Jump type moves, we might prefer to think of a chance move inserted 
Preceding each one of P. It would assign him his detection sphere by 
choosing its center equiprobably in a sphere of centre E and radius r. 

For the moment adhering to the discrete version, the information that 
P receives may be as indicated in Figure 12.2.la. The dots are successive 
positions of E, but P only knows that each is equiprobably within its 
containing circle. 


Suppose there comes a time when two successive circles appear as at (b), 





i > 


Figure 12.2.1 


. Except possibly some relating to the starting position. 

* Our illustrations do not suffer if we leave E’s information about P unspecified. 

‘Ina space whose coordinates are all the state variable pertaining to E (his position 
and possibly his direction, speed, etc.). 
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where AB is only very slightly less than the span available to £ in his last 
move. Then P knows that E must be very near B. This phenomenon need 
not be uncommon. The special case where E is stationary will, in time, 
result in something like (c) of the figure; if P waits long enough, with 
probability one, he can infer as exact information as he pleases. 

Thus the inferences of accumulated granted information may be much 
greater than the current allotment. This type of phenomenon again explains 
why a good strategy must depend on past states. E 

The opulent overlapping of spheres, as at (c), is in part due to our 
stipulation that they are probabilistically independent. But what happens 
if we take finer quantizations in an effort to approach a continuous game? 
We would come ever nearer to certain determination with ever shorter 
intervals, ultimately tantamount to no error at all, to a game with full 
information. 

A remedy for such an absurdity emerges if we look at actual con- 
tinuous measuring instruments and the science of their errors, a part of the 
theory of stochastic processes. Common in this subject are autocorrelated 
functions which have small probabilities of large changes for small in- 
crements of their arguments. Obviously a constraint of this type is needed 
for realism in the continuous game. 

There is an analagous unreality in strategies. It seems clear the optimal 
play will be mixed in games with essential limits on information. Yet how 
is the randomization of a control variable to be effected? At each instant, 
to allow a player the choice of a value from an independent probability 
distribution smacks of an absurdity similar to the above. For any 
realistic model, the continuous choice, say, of a rudder position demands 
that near successive positions be correlated whether the agent be man or 
mechanism. Again, it seems, we must speak with a stochastic accent. 

U. Grenander, in a splendid work of eighty-four pages,® has shown the 
way. He deals with pursuit games which have a steady state character, and 
the decisions of the players are proscribed by stochastic means such as 
appear in prediction theory. 

"Iet us Susi to the Sarat game and spectulate as to what the full 
solution may be like. To illustrate possibilities we will make the artificial 
assumption that r, the detection sphere radius, is constant’ and large in 
comparison with the capture radius. 

If the partie begins with the distance PE much greater than r, we can 
expect the early play to be like that of the game with full information; P 
will pursue the relatively small sphere as he would E and E will employ 
corresponding evasive tactics. But with proximity, especially after P enters 


ê Reference [12]. f i 
7 Instead of diminishing with proximity as would be likely in practice. 


[12.2] A SPECULATIVE PURVIEW 341 


the sphere, the lack of information will introduce a searching and hiding 
phase. If we suppose a very high autocorrelation so that the detection 
sphere is virtually stationary and that, once P gets within it, he so remains 
without much sacrifice of agility, the ensuing play will be virtually a 
search game with the interior of the sphere acting as &.8 

This pure search aspect fits our pursuit game only if P has such great 
kinematic advantage over E that the latter can be regarded as relatively 
stationary.” But in other, and probably more general, cases P will be 
obliged to divert some of his effort to keep up with the moving sphere. 
Thus his strategy will be a mixture of pursuing E (say, by pursuing the 
center of the sphere) and the random search as above. Similarly, E must 
blend an evading strategy with his random hiding one. 

In any case, the search game will be a constituent, and we will discuss 
such games in the next two sections. 

Is the transition between the phases—the early pure pursuit and evasion 
and the later one with a blending (or entirety) of mixed searching and 
hiding—gradual or abrupt? 

Now let us ruminate on some of the other five types of partial information 
mentioned earlier in this section. 

If 3 holds, so that P only knows E’s relative bearing, again the knowledge 
of E’s position will accumulate with the passage of time. Were E stationary, 
P, with sightings from only two positions, could spot E precisely by 
triangulation. For mobile E, P will have a harder task, but how much so 
will depend on his knowledge of E’s kinematic limitations, especially his 
speed. 

We can readily envisage strategies of E deliberately designed to frustrate 
P’s garnering of information. They will have to be random, of course, for 
(once again) if P had a forecast of E’s locations he could triangulate on 
them as well as against current positions. 

Under 5, with P’s sightings intermittent, periodic, and widely spaced, we 
can muse on the advisability of E’s mixing over courses equiperiodic with 
the P’s sightings as in Figure 12.2.2. The deception ensues from P’s 
spotting E only at the points marked by large dots. 

Situations with a time lag 4 are commonplace. For example, feinting is 


° At this point we can convince ourselves of the need for random strategies. In a 
search game P and E within a sphere &, suppose P had an optimal pure strategy. It 
would direct him to search the portions of £ in a certain order. But E being able, too, 
to compute this strategy, need merely always stay in a part of £ remote from P. Similarly 
a pure optimal strategy for E would supply P with an advance schedule of E’s positions. 
Each player can only outwit the other through “mixing”—picking one of a variety of 
routes through chance. 

°? That is, if we take coordinates centered at E, the motion of the sphere hinders P 
but slightly and he is free to search it. 
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Figure 12.2.2 


effective only when there is a lag between the opponent’s observation and 
his active response. Let us reconsider the earlier football examples with a 
ball carrier E confronted by a tackler P. A feint by EZ, such as a lunge to the 
left, followed by a quick veer around P’s right, would be futile in its 
attempt to throw P off balance if the latter were capable of instantaneously 
reacting to E’s movements. i 

We have discussed a number of ways the players might move for the 
sake of information—P to gain it, E to deter it—which almost always 
entail randomizing and have suggested that, in pursuit games, optimal 
strategies should be a blend of these type moves and those of more direct 
pursuit and evasion, resembling (at least) the optimal strategies were the 
information complete. Now it is not certain that such will always be the 
case. The former type of moves—those concerning information—generally 
penalize the latter—those of direct pursuit and evasion. There will be 
instances when the penalties are too high—the example below is an 
extreme case—and another basic facet of the general problem is to find 


criteria. 


Example 12.2.2. The simple pursuit game. We revert to Example 1 9.1 in 
which P and E move in the plane, each with simple motion, P having the 
greater speed. The payoff being time until capture, we know that optimal 
play consists of P’s chasing E along the straight line through their starting 
positions. Now suppose P has only information of type 3; he only knows 
the relative bearing angle of E, not the range. But the bearing angle is all 
he need know to play his optimal strategy of the game with full infor- 
mation. Obviously this strategy is best here, too. Neither player should do 
mixing. 

af sould be possibly advisable for P to veer off course to triangulate 
only if the game were altered in such a way that the extra information a 
advantageous to him. We leave it to the imaginative reader to devise suc 
alterations. 

Similar, but less extreme, ideas apply to our other cases. For example, 
the oscillatory routes of Figure 12.2.2 will curtail E’s effective evasion 
speed. With a payoff such as capture time, it is doubtful whether he 


should employ them. 
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But if we shift our thinking from pursuit to firing type games, the 
situation changes. Here P is equipped with a weapon such as a gun 
(torpedo, missile), and he may fire one or a succession of shots at E. 
Let us first ignore any inherent inaccuracy in the weapon, so that P is 
certain to score a hit if he knows E’s location exactly. The payoff is the 
hit probability. 

Then, as E’s only objective is to degrade P’s information, it is clear that 
randomized strategies are essential. Similarly, P will have to mix, for any 
set policy of when and where to fire would enable E to be not at target 
location at the time. 

A generic instance of such games of great practical importance illustra- 
tive of these ideas is embodied in 


Example 12.2.3. The problem of delayed firing and evasion. An instance 
of 4, there is a time lag between P’s sighting of E and the arrival of the 
projectile. Let us say P is allowed one shot, which he must aim at some 
future location of E. The objective of E, assumed mobile, is to maneuver 
so as best to confound P’s prediction. Randomization is the essence, for 
any systematic zigzag is as predictable as uniform motion. But just how is 
a difficult matter, for we assume that E knows nothing of the shot until the 
arrival of the projectile and P is free to fire at any time. It might appear 
that at any instant he minimizes P’s information with a mixed strategy that 
results in equiprobable distribution of E’s location over all possible places. 
But not knowing when the blast will come, he ought be equiprobable at all 
times and this is impossible. 

Of course, one can coin many explicit games of this ilk. One such, the 
simplest nontrivial one possible, will reappear as Example 12.6.1. 

Pure aiming and evasion games, such as the previous example, which 
have a sort of “stationary” character (see Section 12.6), and the objectives 
can be expressed in terms of information alone, are grist for the theory 
of stationary stochastic processes. The extant techniques of optimal 
prediction can be utilized for P and reversed for E. That is, the latter 
player seeks that haphazard course which maximizes the prognostic error 
of his whereabouts. Grenander’s reseaches in this direction are excellent 
and deep and very likely point to a future theory that is complete, elegant, 
and useful. 

But once stationarity is waived, our ideas cloud again even when the 
payoff embodies prediction alone. 

For example, let us waive our assumption of the absolute accuracy of 
P’s firing and suppose that it diminishes with range, the distance of E 
away. Then it seems that E’s random strategy must incorporate, to some 


10 For an extended explanation, see the fourth of References [13]. 
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extent, fleeing from the weapon site. To what extent? Can the rate of 
decrease of accuracy with range become so critically great that, when 
exceeded, E’s optimal strategy is the pure one of just fleeing? 
Another variant is had by adding a destination for E; once there, e 
mission is accomplished; P’s objective is a prior hit. Let us so modify 
Example 12.2.3 and also suppose P to have more than one bullet. When 
E is fairly close to his target, going there directly makes his aa 
perfectly predictable. Yet using random motions to disguise it keeps l » 
vulnerable to fire longer. Again the dilemma of fusing a pure and mix 


strategy! , aa i 
jea instance is a simplified version of a case of vital importance: 


12.2.4. Invader interception following early warning. An enemy 
PET guided missile) Æ is detected while still far from a known 
target. Its speed and heading as well as its location are known at some 
instant, as might be the case with a DEW (Distant Early Waring) me 
Defending interceptors instantly are launched; oneis shown starting from 
in Figure 12.2.3, W being the point of detection. A naive defensive strategy 
would be the straight collision course based on the assumption that E 
persists in a constant velocity. The dashed paths in the figure show the 
ensuing capture occurring when P is at C which is surrounded by a circular 


capture region. 
But what if E takes a less direct route to the target? The solid paths 


sketched suggest some possibilities (for emphasis, let us not fear fantastic 
deviations). If E randomizes over some set of these, the interceptors 






Target 


Figure 12.2.3 


11 We suppose no fuel limitation on E. 
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are faced with what is essentially a search game. Their object is to discover 
E before he reaches the target. 

We cannot solve this game whose optimal strategies are certain to be 
mixed. But it is another illustration of the blending quandary. Use of 
extreme, outré paths by E will make finding him difficult, but their great 
length leaves him vulnerable too long and so deters his probability of 
success. How best should he compromise between the pure—going 
directly to target—and the mixed strategies ? 


12.3. SEARCH GAMES WITH IMMOBILE HIDERS 


In the simplest case one player E, the “‘hider,” secretes an object some- 
where in a region &, which may be in a space of any dimension number. 
His opponent P, the seeker, strives to find it in the least time. As E 
endeavors to maximize this time of search, it is the payoff. 

We shall suppose simple motion on adequate approximation to P’s 
mode of travel—a fixed speed with complete freedom in his choice of 
direction. Surrounding P will be a fixed region of surveillance, let us say a 
sphere of diameter d. The object is found when it is contacted by this region. 
Then, as P searches, he cuts a swath of diameter din Z. A traverse of 2 
by P in which all of Z is searched without duplication will be called a tour. 

We are going to be tolerant in this definition. For example, with a 
circular surveillance region, P could not execute a perfect tour of a square 
Z. But we shall overlook small transgressions, such as the overlaps that 
occur on sharp bends of P’s path or small protrusions outside of 2 due to 
serrations or sharp corners in its boundary. Thus we shall suppose that 
the cross-section “area” of P’s swath times the length of a tour is the 
“volume” of Z. (The words in quotes apply to a three-dimensional Æ; 
for a planar one read “width” and “area”). Consequently, the length of 
tour is fixed, and, as P moves at a given speed, so is the time required. The 
latter we will call T. In practice, a sensible complete search of Æ without 
unavoidable duplication or waste should require but little more time than 
T and we neglect this excess. 

In this simple search game E has one move: he places the object 
anywhere in Z. Then P endeavors to find it as quickly as possible, starting 
from any point of & he wishes. 


THEOREM 12.3.1. The Value of the simple search game is $T. The only 
optimal (mixed) strategy of E is to place the object equiprobably in Z. 
An optimal strategy for P is to traverse some tour and its reverse (the 
same route traveled in the opposite direction), each with probability }. 

Proof. Let us think of the route of some tour unfurled into a straight 
line. Then, to within a reasonably small error, each hiding place E might 
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select in Z can be identified with the point of the tour (line) at which the 
object is discovered. 

1. Let E play the equiprobable strategy. If P searches via any tour, 
then the object’s location will have a uniform probability distribution over 
the line. As P traverses this line with constant speed, the expected time 
to discovery is one-half the total or 4T. If P elects any search scheme 
not a tour, it is clear that its inefficiency will render the expected 
time > 3T. 

2. Let P play the strategy of the theorem. A pure strategy by E is 
tantamount to his selection of a point on the unfurled tour. If U is the 
resulting time of discovery for a traverse in one direction, this time will be 
T — U for the other direction. The payoff is then 4U + (T — U) = }T. 
The theorem follows from the standard definitions of game theory. 

Remark. We see from the proof that the liberty granted P of choosing 
his starting point is far more generous than necessary. 


Problem 12.3.1. -Give an optimal strategy for P essentially distinct from 
that of the theorem or a mixture of such for different tours. 


If P governs a team of s identical searchers, it is evident that he should 
divide Æ into s portions of equal area and assign one searcher to each. 
Optimal play for P results when each searcher acts optimally in his portion. 
Thus we have 


COROLLARY 12.3.1. The Value of the simple search game with s searchers 
(and one hider) is $7/s. 


On the other hand, if the number of hidden objects is augmented, the 
payoff being the time required to find them all, the difficulty of the problem 
is vastly increased. Ascertaining the optimal strategies is a recondite 
business and they can depend on the shape of &. For example, if Z were 
long and narrow, with cross-section diameter <d, the only tours possible 
are the two which go from one end of 2 to the other. With one searcher 
and two hiders, E’s best strategy is to place the latter at the extremities of 2 
for the payoff will then be at least 7, which is the best possible. For a 
spherical 2, on the contrary, it is evident that no pure strategy can be 
optimal for E. Even more difficult are optimal search strategies. 

But—and this our main point—if the number h of hiders is at all large, 
the searching strategy matters little as long as it entails no duplication. 
That is, P may allocate to his s searchers each a tour over 1/sth of Z. Even 
if he announces the full strategy and E exploits this knowledge by hiding 
in the last places the searchers will look, the payoff will not be greatly 
increased over the Value. 
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More precisely, we shall essentially prove 


h+i1s © > 
where V is the Value. The inequality shows that for A reasonably large 
say 10, V is not far from Tjs, the duration of a cooperative tour. Thus an 
such tour will net P a payoff close to the Value. 

We shall derive the counterpart of (12.3.1) for a discrete model. Let 
A be peppered with dots, contiguous ones being connected by lines; thus 
it is approximated by a linear graph. The new version of our game is the 
obvious one: E secretly places the h hiders on any distinct dots; P in 
turn starts the searchers at any s points and at each move shifts each to 
an adjacent (line connected) point. A hider is found when a searcher 
occupies its point. The payoff is the number of moves which finds them all. 

We shall take it that a sufficiently fine and regular graph simulates the 
continuous game closely. It is convenient and harmless to suppose that 
the number N of points is divisible by s. 

The result on discrete games, Theorem 12.3.2 below, will be taken as 
adequate evidence for (12.3.1). 

Let v be the payoff when the strategies are as follows: E i- 
probably, that is, the hiders are distribui randomly with E Ar 
h points equally likely; P plays any definite multiple tour, that is, no 
searcher ever moves to a point previously searched. i 


T 
a (12.3.1) 


LEMMA 12.3.1 


Proof. If all hiders are found on the kth move, P will have covered ks 
points. The h hiders must have been at these points with at least one 
occupied that was explored on P’s final move. The number of ways of 


placing h hiders on ks points is (7): We deduct the unwanted cases 


bri iy hiders are on the (k — 1)s nonfinal points, which number 
— 1)s ; PE? 
h ). The difference, when divided by , the total number of 


hider allocations, is the probability that the payoff is k. As v is the ex- 
pectation of the payoff 


= x fa _ ae Ds) 


=E (i) 


where M = NJs. 
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By rearranging (“summation by parts”), 


M (=) — 0 
-Al 
where 


As Ms =N, 


To assess o We utilize the familiar relation 
nf _ (n+ i 
3 (1) = (ha) 
j j j+1 sj +1) —- ‘) 
s(t) < (A) + (UE a Os 


h 
which is obvious if the left side is regarded as a sum of s arame = 
sum over j and note that “numerators” on the right are totally 


secutive integers from 0 to sM — 1 = N — 1, we have 


Finally, 





with N points of the simple 
the Value V satisfies 

N 
h NeyeH. 
h+is s 


i in a multi 
Proof. As N/sis the number of moves inam 
must ieee all hiders, the right inequality 1s clear. 


THEOREM 12.3.2. For a discrete version 
search game with h hiders and s searchers, 





ple tour, which certainly 
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Let E play equiprobably. Any tour by P leads, by the lemma, to a 
payoff > (h/h + 1)(N/s). This inequality applies for all strategies of P, 
for any such with duplications could be replaced by a better one without 
them. Because E is the maximizing player and has a strategy guaranteeing 
a payoff satisfying this inequality, V must satisfy it too. 


Research Problem 12.3.1. Analyze games like the foregoing except that 
they terminate when a certain number (< A) or fraction (< 1) of the hiders 
is found. 


12.4. SEARCH GAMES WITH MOBILE HIDERS 


As far as we know, virtually nothing is known about the solution of 
such games. The following instance appears to embody the quintessence 
of the problem. 


Example 12.4.1. The princess and the monster. The monster P searches 
for the princess E, the time required being the payoff. They are both 
in a totally dark room & (of any shape), but they are each cognizant of its 
boundary (possibly through small light-admitting perforations high in the 
walls). Capture means the distance PE < /, a quantity small in com- 
parison with the dimension of 2. The monster, supposed highly intelligent, 
moves with simple motion at a known speed w. We permit the princess 
full freedom of locomotion. 


We do not know how to solve this problem, but it seems certain that the 
optimal strategies will be highly randomized. Our feeling is that just how 
the players use chance to pick their paths is secondary; probably any 
haphazard meandering will do about as well as any other. 

We conjecture that the sole decision of importance rests with E. How 
fast should she run? One extreme—complete immobility—seems un- 
promising. For P, by a tour of Z, can be certain of a payoff not exceeding 
the fixed duration of such, whereas any sort of motion by E offers at 
least the possibility of an arbitrarily long period of freedom.’ At the 
other extreme, a very high speed (compared to w) by E can hardly be 
desirable, for in a short while capture is nearly certain; she will run into 
the monster. 

Somewhere between the extremes, then, will be an optimal speed for E. 
Where? 

This example typifies the problem of pure search games. The following 
one is similarly unsolved but appears. simpler and may possibly serve as a 
stepping stone. 


18 Of course, it may be that the expected period is less than the time for a tour, but it 
seems unlikely. 
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Example 12.4.2. A simpler princess-monster game. The only innovation 
is that now P and E are each confined to a closed curve. We take it as a 
circle. 

One simplification is that at each instant, if their speeds are assumed 
fixed, the players each have but two directional choices. 

The conjecture as to E’s optimal speed now appears to have a natural 
candidate: E should employ w, P’s speed. Our shaky grounds are merely 
that equal speeds alone preserve the distance PE should both players use 
the same pure strategy. 


Research Problem 12.4.1. Solve the discrete version of this game. The 
players each occupy one of n (> 3) points distributed on the circumference 
of a circle. They move alternately, transferring their positions to either of 
the two adjacent points. Capture occurs when either both players occupy 
the same point or they occupy adjacent points and interchange positions 
after one move apiece. The payoff is the number of moves, say, of P 
until capture. An equiprobable distribution is probably the most desirable 
way to get the play started. 


12.5. THE IMPORTANCE OF APPROXIMATIONS 


Almost all the investigations on game theory up to the present have 
been concerned with precise statements, but there are essential domains in 
the subject where the exact solution has but a negligible practical advantage 
over an approximation thereof. Of the two chief such categories we discuss 
below, the first pertains not only to games but to the whole general field of 
maximization.1*> The second applies to games of incomplete information 
whose solution entails a mixed strategy. 


1. The principle of flat laxity 

Although we have ventured to affix a name to this principle, it is so 
obvious that there is embarrassment even in its mention. It lies under the 
nose of every student of the elementary calculus, yet we can recall no 
instance of its explicit delineation. 

The student learns early to maximize a function by seeking points where 
its derivative is zero. The obvious principle we wish to asseverate is merely 
this: 

At such maxima the value of the argument of the function is generally not 
critical. This is simply because the derivative—interpreted in the elemen- 
tary way as a rate of change—is zero. Elementary as such ideas may be, 
three types of maxima are shown at (a) of Figure 12.5.1: one with zero 


13 As is well known, logically equivalent to minimization, for we need but 
maximize the negative of a quantity to minimize it. 
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derivative and two other common types, the third being at the endpoint of 
the domain of the argument. Observe the relative changes in function 
value for the same deviation of the argument from its value at the 
maximizing point in the three cases. 

Solutions in the variational calculus which are integrals of the Euler 
equation likewise often belong to the zero derivative genre; we shall call 
the whole class of this type of maxima (and minima) flat. They are all 
subject to the title principal: small deviations of the argument from its 
optimizing value are uncritical. 

It is highly likely that the flat laxity principle accounts for the aston- 
ishing dearth of practical applications of the many elegant mathematical 
maxima that have appeared throughout scientific history.14 

This discussion can be applied to saddlepoints and so to game theory. 
In differential games, the principle supplies some pragmatic advice. In 
those cases with interior maxima or minima of the control variables, the 
gains from a too scrupulous adherence to an optimal strategy may not be 





(b) 


Figure 12.5.1 


1 Examples: The right circular cylinder of a given volume has minimal surface area 
when the height is equal to the diameter. However, tin cans of this proportion are 
seldom seen on market shelves. 

Munk’s elliptical wing planform of minimal induced drag seldom appears in aircraft 
design. 
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worth while. But in those cases with extreme optimizing values, such as 
those with linear vectograms, the gains from optimal play can be very 
genuine. 

Finally, let us look at games of incomplete information with mixed 
optimal strategies. For simplicity, we think of a case with a finite, discrete 
matrix. A typical mixed strategy solution demands that one player play a 
certain subset! of n of his strategies with positive probabilities p,,.--» Pn 
(Zp, = 1), his opponent doing the same with g,,.--+%m- The positivity 
of the p; and q; implies that each is an internal optimizer and therefore of 
the flat type. The principle here grants tolerance for small errors. For 
example, a plot of the payoff, when m = n = 2, as a function of the mixing 
probabilities p and q (we use p, 1 — p, instead of pı, Pa, etc.) might appear 
typically as in Figure 12.5.1b; note that at the saddlepoint the surface has 
a flat tangent plane. 

But matters can be otherwise for strategies not in the foregoing subsets, 
For such the p, and q, (see the footnote) are zero and so are extreme 
optimizers with possible sensitive responses to change. This is a common- 
place rather than a pedantic matter, for a complete game matrix includes 
all strategies, even the absurdly bad ones. The latter are rigidly eschewed 
by even mediocre human contestants. A bridge player does not waste his 
trumps, not even with a small positive probability. 


2. Probablistic indefiniteness 
Again we suppose a game of imperfect information with mixed optimal 


strategies (or at least for one player). The actual usage of such strategies 
makes each partie a game of chance. Changing the strategies shifts the 
odds. 

The second argument for the adequacy of approximate solutions is a 
facet of an old and basic question in the realm of probability. What is the 
actual effect of the probability distribution over a set of alternate outcomes 
when the number of trials is small? 

If a gambler is betting repeatedly on a certain event and if the prob- 
ability of his winning is actually .45, although he thinks it is .50, it will 
take him a great many bets to detect his error. The science of statistics is 
largely concerned with inferences drawn from such repeated trials and the 
subject is not simple. We must not attach unwarranted importance to the 
law of large numbers, which here asserts that for a very long series of bets 
the gambler is almost certain to win very close to 45% of the time. 


18 We wish to stress the fact that this is very often a proper subset; certain strategies 
not being used at all. Such cases can, and usually are, subsumed under generality 
by assigning zero probability to the latter strategies, but we have grounds here for 


distinguishing them. 
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The gambler might be a player in our subject game” and the shift in hi 
win probability due to his opponent’s improving his mixed strategy. F. wee 
small number of plays, then, what value a close calculation of an imal 
ene optimal 

For games with mixed optimal strategies, both principles 1 and 2 c 
be effective and so the two effects are compounded. In certain extre fe 
cases the composite might even make the range of approximation so b ad 
that game theory has not much utilitarian to offer. The practical aly 
gee have a way of recognizing such cases. aaa 

t us take as an example a moving target, say an aircraft i i 
fired on by the enemy. As a game, the Saral at express Sie a 
he scores a hit. The target has certain maneuvers at his disposal maki 
his position hard to detect or predict. This choice constitutes his strate 3 
which in this game, whose essence is information, will certainly be mi a 
ii Aea payoff will be the hit probability. ? a aes 

Suppose that by an improvement in his stra 
hit probability by, let us say, 5, 10, 25, or Oe PE E = 
instances, the situation is like that of the gambler. Willa reduction i he 
odds by 5 or 10% matter very much? For 50%, yes. For 25%? Ai 

But there may be repetition. During a war there may be numerous 
UAE = in ~ weapons; the hit probabilities approach the 

ets destroyed. Is i i 
aircraft or shine lost by 3 %? By Da: ra - a EE ESES 
Such are our grounds for the conclusion that the practical goal of gam 
theory should be, in certain cases, especially those with sparse eE i 
approximations. There seems to be little proof in support. In fact th 
sven game with many immobile hiders in Section 12.3 is the only insta 
r now, In Section 12.4, we conjectured that, with mobile hiders, all 
the steering aspects of the strategies are largely irrelevant. A proof of thi 
z similar results, where they are true, would mark a decided and useful 
$ berate in game theory. Thus one essential problem is somehow to assess 
z egree of approximation appropriate to various classes of games. 
ee Sai ae hoe te cas ee a approximation will be the thing 
r x we find it? We must be w i 
of such subjects as mathematical physics. For we are eta a 
re there is always the opponent ready to exploit to the uttermost an 
eviation from the optimal. As the whole theory is predicated on his 
Bok e A we must posit that he will do so. ° 
„1 one player, for example, is defending a city wi 
his strategy consists of an allocation of his forces EEA BE i 


1¢ Which must be a win-or-lose t “ 
Gie aa aon, ee ype (two-valued payoff) for a perfect paradigm, but 
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have a close approximation to an excellent strategy if he adequately 
guards ninety-nine. But if his opponent has good intelligence, all his 
forces will surge through the neglected gate and the near perfect defense 
is useless. 

So the techniques of approximation will require some innovation. They 
must be proof against the worst advantage the opponent can take of them. 

Although we would encourage the investigation of approximations, 
we do not advocate no other approach. If exact optimal mixed strategies 
were found for, say, the search games of Sections 12.3 and 12.4, although 
their actual use may gain a contestant little, their discovery would probably 
throw a floodlight on an area of dark ignorance. 


12.6. THE CHANCIFYING METHOD 


The name with which we have dubbed this technique of solving certain 
games has a clumsy ring. But the same adjective perhaps fits the method 
itself; however, for a certain class of models we know of no other. 

Suppose a zero-sum two-player game of incomplete information has 
“steady state” or “stationary” character. This concept is defined most 
readily if we adopt a discrete model. The game is stationary if the decision 
pattern recurs cyclically and at each cycle either the partie terminates or 
the situation, aside from the choices made, is the same as during earlier 
cycles. That is, a spectator who began witnessing a partie in midplay 
could not tell how long it had been going on. 

Let us say that P, the minimizing player, moves first. If necessary, we 
must attach an artificial “past” so that the opening position is indistin- 
guishable from one in midplay. We alter the rules so that instead of P 
making the first decision, it is done in accordance with stipulated prob- 
abilities 24,..., w» ($z; = 1) indicated collectively by x. That is, the 


first move is now a chance one. The Value of this new game then depends 
on x and will be denoted by ¢(). 

By applying the rules to the first full cycle of moves, it is often possible to 
obtain a functional equation that ¢ must satisfy. Ifits solution is essentially 
unique so that ¢ can be ascertained, the Value of the original game will be 


V = min ¢(2).” 


For the sole distinction in the original game is that P has the choice of x 
and he will exercise it to minimize V.* 

17 Sometimes inf (x), for there are cases when a player has no optimal strategy, 
that is, he cannot attain the Value V but has strategies that guarantee payoff arbitrarily 


close to V. 
18 For a further instance see Reference [14]. 
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Example 12.6.1. A simple aiming and evading game. This game is a 
discrete model of the type of Example 12.2.3, in fact, the simplest possible 
such that is not trivial. In Section 12.7 we shall briefly return to these 
games. 

A counter rests on one of a doubly infinite row of points. At each of 
his moves P has the choice of moving it right or left to the adjacent point 
The moves alternate with those of E, who at each turn has the choice of 
ee in eae “firing.” If E elects the latter choice, he 
se point. The play then termi 3 i i i 
pian | pees . 7, 2 inates; if the counter is on the point 

Basic is the incomplete information structure—the time lag. Just prior 
to his turn E knows all moves of P except the two most recent. Thus when 
he fires E knows that either the counter is at the last observed point (P 
moved RL or LR) or it is two points to the right [left] (P moved RR [LL] 
And if rational, he will fire only at one of these three points. 
i When a P pay mixed strategies, which we may assume, the payoff 
ecomes the probability that E i i 
aa ne P sey scores a hit, that is, guesses correctly the 
Let us chancify on P. Suppose he normally has the opening move. We 
imagine that P has had a preceding move, say from the left. We replace 
P's move by a chance one which is to the left with probability x and right 
with 1 — z (0 <2 <1). The Value, the minimax of the hit probabilit 
we will call (x), which satisfies A 


xe 
ae x1 —c) +(L— al —d 
#2) = min max | yy y =DD 10.6.1) 


zele) + (1 — x)¢(d) 

Here the max applies to th i i ile mi 
Ms rere aE i the four lines on the right, while min ranges 

To establish (12.6.1), at least heuristically, first, let 
following the chancified one as follows: If ihe counter a ed to lef 
(probability = x), he next moves left again with probability c, and right 
with 1 Te Similarly, if the chancified move was to the right he continues 
right with probability d. The quantities c and d are to be part of P’s 
optimal strategy and will be fixed later. 

The four lines after the brace correspond to E’s four possible responses 
If he fires at the leftmost point—two spaces left of the last observed 
position—he scores only if the counter is there, such requiring two left 
moves, and the probability of this and so of a hit is xc. The next line is 
similarly the probability of the counter’s being at the central point, the 
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$ 


Figure 12.6.1 


preceding moves having been LR or RL. Thus the hit probability, if E 
fires at the central possibility, is the second line in the brace. The third line 
corresponds to E’s rightmost firing. 

Finally, the last line is the payoff if E waits. Had the counter moved left, 
with probability x, we are faced with a recurrence of the original situation: 
the start of a chancified game with c replacing v. Its Value is thus ¢(c). 
As with probability 1 — x, E likewise confronts a game with Value ¢(d); 
the fourth line is his expectation of a hit if he waits. 

Now for any given c and d, E will select the maximum of the four 
entries. Then optimal play will dictate that P choose c and das to minimize 
this max. Such will yield the Value, which is $(2). 

The solution ¢(x) to (12.6.1) turns out involved. In a central segment 
AB of its graph (Figure 12.6.1) ¢ is constant, but in both extreme sections 
the graph consists of infinitely many straight segments whose endpoints 
have limits at A and B. 

We have solved a few other similar problems and resulting ¢ was 
similarly intricate. It’s ascertainment was tedious. And all we need of it 
to solve the original game is its minimum! 

But is there not a hint here of a possible technique of approximation? 
The polygonal portions of the graph in Figure 12.6.1 strike the eye as being 
close to smooth curves. Is there an approximate method such that the 
counterpart of ¢ which it embodies is some simpler function? 

The above game is one for which the arguments for approximation 1 
and 2 of Section 12.5 both hold. Hence, for both it and various more 
realistic versions of the same genre, some of which are of considerable 
practical importance, a well-reasoned and sensible approximate solution 
would suffice and be of great value. 

Further details on this game appear in References [13]. 





Appendix 


Dent ea dias eae the appendix consists of a diverse selection 
ples of differential games. The solutions 

furt i : are sketched 
with hints for aid when unusual features or formal difficulties occur i 


A1. A HIT PROBABILITY PAYOFF 


In certain military games, such a i i 
f s those in Sections A2 and A5 
player, say E, continuously fires a weapon at his opponent. The pa ‘offs 
to be the total probability of a hit. ile a 


The hit probability density p i i i 
l sity p is supposed given: p is a function of t 
ee Nae m conceivably of the control variables also). Thus ie 
e partie, p becomes a function p(t) of the ti ili 
of a hit in the interim (z, £ + A) is Bo A OMD. a ia a 


LEMMA Al. In the above circumst i 
sayin mae stances the game has an integral 
G = p. (A1.1) 
Proof. Let Q(t) b ili is mi 
Pits he x Q(t) be the probability that the target is missed by all fire 
The probability that the target is missed during (0, t + h) is the 


product of the probabilities that it i i 
ist eea it was missed during (0, t) and (t, t + h). 


O(t + h) = QA — hp(s) 
Q(t + h) — 
i et = OO = aD 
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which in the limit is 
Q' = —pQ. 
As Q(0) = 1, we have 


o0 = exp (= [P69 du). 


Thus the probability of a hit during (0, £) is 1 — Q(t) and so is an 
increasing function of the integral. Then if a strategy renders the integral, 
say, maximal, it likewise so renders the hit probability. 


Note that the formal Value V(x) yielded by the solution is not the hit 
probability, the actual Value. The latter is 1 — exp (V(x). 

Let us think of the continuous fire as (it actually is) a succession of 
small shots. It is not hard to show that in two-dimensional space, the hit 
probability of a single shot varies inversely with the range, the distance 
from weapon to target, for reasonably large r. Thus p may be taken as 
ajr for some constant a. In three-dimensional space, correspondingly, 
p =ajr*, (The former is used in the two succeeding problems only because 
it facilitates the formal mathematics.) 


A2. THE FIXED BATTERY PURSUIT GAME 


Both players have simple motion in a plane. The pursuer P is faster and 
could capture E readily were he not deterred by a barrage of gunfire as- 
sisting E. The firing is continuous and emanates from a battery located 
at a point O. The instantaneous hit probability is inversely proportional 
to the distance OP, and its time integral, as in Section Al, is to be the 
payoff. 

Thus P seeks to capture with a minimum chance of being hit; he must 
choose a course which compromises between a direct chase of E and a 
wariness of being too close to the battery for too long. Similarly, E must 
inculcate into his flight from P maneuvering which lures P into lethal 
proximity to O. 

We shall use the polar coordinates r, 0 for P and the rectangular co- 
ordinates x, y for E, both systems having the same origin O with the lines 
6 = O and y = Ocoinciding. The speeds of P and E shall be 1 and w with 
w < 1, so that w is really the speed ratio of E to P. The control variables 
¢ and y are as in Figure A2.1. The KE are thus 


F=cos¢ 

6 = I/rsing 
$ = W cos y 
ý = wsin y. 





[a2] 
The payoff is integral with 
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G=? 
r 


a being a fixed positive constant. The terminal surface is the set wher 
|PE|=la given positive number, and @, the set with |PE| > 1. j 
The solution is obtained almost entirely by pure integration “this example 
being very exceptional in its freedom from singular phenomena On 
singular surface there certainly is: when P, O, E lie on a straight line in 
this order, the two symmetrical ways that P may skirt O see impl 
dispersal surface, with an instantaneous mixed strategy. ae 
Our standard method then presents no difficulties (except possibly the 
common one of deciding the sign of the V, of the initial conditions), From 
the RPE, it will follow at once that E always travels a straight ath A 
route of P, however, lies on a curve: ce 


1 ; = 
ae cı sinh (+,/K(8 — cs)) + c, cosh (+./K(0 — Cs) + cy 


where c,,...,¢,, K are constants; K m i i i 

Cay ; ay be of either sign, meaning th: 
for some paths sin, cos, replace sinh, cosh. E gii 
, Seme A2.2 is a scale plot of typical partie. Here w = %; the dots 
indicate Corresponding values of r. Naturally any corresponding pair 
may be considered as a starting position. 


Exercise 42.1. Write the KE for this probl i i 
Rr ried problem using three state variables 


oe Solve a version of the preceding problem which differs 
rom it in that the battery, instead of being at a point, is distributed 





Figure A.2.1 
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Figure A2.2 


uniformly over a line Z. That is, G = ald, where d = distance P 
to 2. 

Research Problem A2.1. Suppose in the problemi of the text we had taken 
for G a function more rapidly decreasing than the inverse of r. Intuitively 
it might appear that, were this decrease rapid enough and had E se 
enough, the best strategy for E would be to get near O and to loiter : ere. 
Is this ever actually the case? If so, how is such a problem solved? 
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A3. OPTIMAL TRAJECTORIES OF GUIDED MISSILES 


The design of long-range missile trajectories so as to maximize efficiency 
is a task amenable to the present methods. We are faced with a one-player 
game, the payoff or quantity to be optimized usually being the fuel 
consumption. 

Let us look at a prototype case. The coordinates of &, the state variables, 
will be such quantities as 


the positional coordinates of the missile 

the velocity components 

the yaw angle and other such inclinations 

the current weight of the missile (decreasing due to fuel consumption). 


The control variables will be, of course, just those quantities regulated 
by the guidance system. 

Let us say we are dealing with an ICBM designed to go from one point 
on the earth’s surface to another distant one. It will have two or more 
stages of powered flight, possibly jettisoning some of its bulk at the end of 
each and concluding with a free fall stage. 

In our approach to the problem, we work as usual retrogressively. 
Starting from the given destination, we ascertain the set of state variables 
enjoying the property that free fall from any such condition will bring the 
missile to target. The set will comprise a surface @, in £. Using @, asa 
seat of initial conditions, we construct the retrograde optimal paths by our 
usual methods; the result solves the last powered stage. This solution is 
extended back until it fulfills the condition of the preceding stage juncture. 
Such states form a new surface &,, with the Value as a function on it. 
Similarly, we proceed until we reach the stage containing launch. 

Unlike many of the other problems we have treated, this one has enjoyed 
a great deal of attention during recent years. Many analysts have obtained 
first rate results without having heard of differential games. Can we claim 
any advantages for the approach just sketched ? 

Within the pure logic of method, no. 

For one always begins by setting up a model, which is a more or less 
simplified interpretation of reality. It is to the model and to it alone that 
the mathematics is applied. Generally, and almost certainly here, the 
underlying problem will be set so as to have a unique answer. Then all 
techniques which obtain this one answer are equally valid. Indeed, there 
should exist means of rendering them logically equivalent. 

But in regard to the following two questions, differential games has 
advantages over at least some alternate approaches. How much light 
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does the procedure shed on aspects of the general situation other than the 
mere trajectory? Does the underlying mathematics suggest a treatment of 
a more realistic model of greater intricacy? 

First, through the concepts @,, as discussed above, the missile is not 
proscribed to a prescribed path. The vehicle is routed optimally to the 
optimal point of transition to free fall. (Of course, this is also achieved 
through classical calculus of variations with variable end conditions, 
Thus our point seems a minor one: that this consideration is more 
indigenous to differential games.) 

The second advantage of the differential games approach is that the 
missile is not rigorously confined to a single navigational program. Should 
some untoward event put it off course, our craft does not strive to regain 
its old trajectory but assumes the optimal flight procedure pertinent to its 
new conditions. Mathematically this situation occurs because our method 
leads to the calculation of V and hence of the control variables (expressible 
in terms of the*partial derivatives of V) throughout &. This means that 














whatever conditions (state variables) should happen to arise during flight, 4 Figure A3.1 
we know the best way to navigate at these conditions. oe be a vector of length F and inclination $, the latter being the singl l 
: *@ — variable. The KE are in this c i ne single contro. 

Example A3.1. A simplified ICBM trajectory. The simplifications are %@%. motion (g = the usual gravity pea ? mmon Newtonian equations of 
fairly drastic, but nothing of principle is sacrificed. Our model is illustrative : 
rather than realistic. We assume 4 pie 

A single stage of powered flight, followed by one of free fall. ¥ =v 

A flat earth; the gravitational force is uniform and vertical. No air ; ù = Foos $ 

friction. E 

The missile thrust is of constant magnitude; steering is achieved by ` ù = Fsin $ — g. 

varying the direction of the thrust vector. 7 We wi 

The realistic space is two dimensional; the missile always remains inthe -f e will construct & (the @, of the preceding discussion), which is the 











set of all possible points of & where fr i 
ee fall can begin; 
represents the x, y components of a typical one. A 


A little elementary dynamics leads to the representation : 


vertical plane containing the launching site and target. 
The loss in weight of the missile due to fuel consumption can be 
neglected. of 


Thus in the zy-plane this missile is to be launched from the origin O with ; f £ = R — 545, 
zero velocity. The target, labeled R in Figure A3.1, is at (R, 0). The 
missile proceeds under powered flight (solid curve) to the point K where z 
free fall begins. 

The payoff to be minimized is the energy consumed. Due to our 
assumptions of constant thrust and missile weight, this is tantamount to 
minimizing the duration of powered flight. 

The state variables will be æ, y, the missile position coordinates, and 4, 
v, its velocity components (see Figure A3.1). The thrust per unit mass will 


Y = —S253 + gS? 








u= s 


V= 5 > 0,5, >0. 





The reader can easily verify that if a body starts a free fall from the 


above position (z, y) and with initi f f 
iaeio ain 2 a with initial velocity (s;, s), at the time s; it will 
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Forming the ME and RPE, then integrating the latter with the above 
initial conditions, leads to the paths: 
u = sı + FSA — S2 + gSa)T 
v = Sa + (F%sgds, + g)T 
x = R — 5483 — ST — F?SA(— Sa + g53)$7” 





Y = —S2S3 + bgs — sar — (Fĉ°saås, + g)$7° (A3.1) 
where 1 
pi = FsyV/s," + (s2 — 85s)”. 
Also, V = 7 and for the optimal strategy 
tan 6 =——. 
8S3 — S2 


Thus the thrust direction will be constant throughout the powered flight, 
a point to which we shall return later. The trajectory will therefore be a 
parabola with axis parallel to the resultant vector of the thrust and gravity. 

To proceed further involves the process of solving the system (A3.1) for 
Sy» Sg Sg, and r. The elimination leads to a fourth degree equation, and all 
the sought quantities are simply expressible in terms of the proper one of 
its roots. 

But if we are concerned with the particular trajectory of Figure A3.1, for 
which the starting conditions are 


s=y=u=v=0 


we are led to the actual flight trajectory easily. We can easily obtain a 

linear approximation of V in the neighborhood (in @) of this trajectory, for 

the partial derivatives of V will figure in the complete integrals of the RPE. 
Carrying through this special elimination leads ultimately to 


s—Fs?—1)=0 
g 


where S = sin $. Thus the thrust angle is a function of F/g. It is easy to 
plot this relation by inverting a plot of F/g as a function of S. Such is done 
in Figure A3.2. 

The quantity F/g can be regarded as the magnitude of the thrust when 
the weight of the missile is taken as the unit of force. It must exceed 1 in 
order for the craft to leave the ground. When it is very large, the plot 
shows that ¢ is near 45°, a conclusion to be expected, for this is the well- 
known ballistic firing angle of maximal range (in a vacuum) and very 
large thrusts are of very short duration. As F/g diminishes toward 1, the 
thrust angle increases, approaching vertical launch. 





[a3] OPTIMAL TRAJECTORIES OF GUIDED MISSILES 365 











On the same figure we have plotted 7/(r + s3) [= S/(F/g)]. As this 
quantity is the ratio of powered flight to total flight time, it serves a 
poignant descriptor of the total optimal trajectory. With large thrusts, the 
duration of powered flight is small; almost all the route is free fall. The 
two times are equal when F/g is about 1.6, and for still smaller thrust 
the powered phase predominates. 

We saw that in this problem travel in least time implied a constant 
thrust direction. But such is not always the case, even with our simple 


(or no) gravity force field, no friction, etc. From a general integrati 
; , etc. ration of 
the RPE, we obtain : z 


Cyr + Ca 


Crt t, (A3.2) 


tang = P- 
Vu 
where the C; are constants. 

If the object of the craft is simply to reach a given Jocation in minimal 
time, then ¢ will be constant. But the general @ that we have been 
considering entail u, v as well as x, y, that is, terminal velocity as well as 
position count. It is clear that the craft when approaching @ must 
anticipate its ultimate attitude before arrival. The form of (A3.2) is 
plausible in this regard. For when 7 is large (x far from @), ¢ will be 
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nearly constant, but as x nears @, the thrust vector shifts direction in 
anticipation of its terminal conditions. 

We stress this point to indicate that the simple anwer to our problem was 
perhaps more coincidental than indigenous. It involved a constant $ 
even though velocities were germane to @. However, had the problem 
been altered in any of a number of simple ways—air friction in the free fall, 
multistage flight, etc.—it is likely that (A3.2) rather than constant $ would 
optimize. 

Problem A3.1. For a projectile steered by a variable direction thrust, as in 
the preceding example, in a constant force field, show that a necessary and 
sufficient condition that the least time paths to @ entail constant ¢ is that 


on @ VV, — VV, = 0. (A3.3) 


In particular, show that this condition holds if the terminal conditions 
stipulate location but not final velocities. 


Research Problem A3.1. If instead of the constant force field, there is one 
derivable from an arbitrary potential G(x, y)," what are the counterparts of 
the navigational schemes ¢ = constant and (A3.2)? A case of particular 
interest would be the inverse square field. 


Research Problem A3.2. What is the most economical way to send up a 
satellite to any circular orbit about the earth? We have in mind here an 
academic but interesting model. Make the same flight assumptions as 
earlier in this section except that gravity is radial and to vary inversely as 
the square of the distance from the center O of the earth. For © we use the 
set of all conditions at which the missile will be in a circular orbit about O 
(centrifugal must balance gravitational force). 

Supposing the earth perfectly penetrable, under what conditions. will 
the orbital radius be greater than that of the launch point (assumed on the 
surface of the earth) so that the orbit will be a possible one? Even with 
possibility in this sense, the trajectory may still penetrate and then leave the 
earth. If so, it might mean that satellites should be launched from 


mountain tops. 


A4. AN ILLUSTRATION FROM CONTROL THEORY 


We have noted that the modern theory of control seems to be subsumed 
in the topic of one-player differential games. The following simple yet 
typical instance is borrowed from Hale and Lasalle.’ 

1 So that two of KE become u = F cos $ — G,, v = F sin $ — Gy. 

2 Differential Equations: Linearity vs. Nonlinearity, SIAM Review, Vol. 5, No. 3 
(July, 1963). 
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A body of unit mass moves lineally i i i 

i i i y in a fliud of unit viscosity (t 
viscous drag is the negative of the velocity). We may exert e = 
ae : ee to body whose magnitude may not exceed 1 The 

ect is to bring the bod i i in the 

eae & the body to rest at a prescribed point O in the least 

Let x be the positional coordinate, of th i 

5 e body, with O at x = 0 

y be the velocity. The kinematic equations nee are peer 


t=y 
ġ=-y+$, —-1<d<1. 
(The second states: acceleration = visco 
cO : = us force 
payoff is integral with G = 1. ages 


The authors’ solution is shown in Figure A4.1.3 The two curves C are 


sem rf y l y 
P 8 h 8 , 


The genesis of the semiuniversal curves is interesting. Our method 
requires that we take as @ a circle of radius / centered at x = 0 y = Oand 
ys 





Figure A4.1 


* Their F equals our ¢. 


CE ed 
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later let /— 0. With positive / there will be a full family of paths from @ 
with two transition curves. As / becomes 0, the paths coalesce into just the 
two shown. The transition surfaces must coincide with them and so 
become semiuniversal. 


Research Problem A4.1. The same as above except that now we wish to 
minimize the energy needed to arrest the body at O. As such is fọ dx, we 
take G = ¢y, the only formal change. 


A5. THE BOMBER AND BATTERY GAME 


This contest illustrates the astonishing variegation that may be hidden in 
apparently simple military (or other) games. The full solution splits into 
nine separate cases, and even some of these admit of further logical sub- 
divisions. We shall but indicate this panorama here with emphasis on 
some especially instructive points. 

An attacking craft, steered by P, moves in the plane with simple motion 
at speed v. His destination is E’s territory, which is bounded by a coastline, 
a curve X. From a battery fixed at a point O, E fires at the invading P, 
in accordance with Section A1, so that the payoff is the hit probability. 

Each player will be subject to a distinct type of constraint. 

For P it will be the flight time, fuel, or path length, all equivalent because 
of his constant speed. We select the first: total flight time < T. This 
constraint is of the type discussed in Section 5.7. 

For E we shall limit the amount of ammunition m he has available for 
firing at his target P. Let c be the maximal rate at which E can fire. His 
control variable y will be the fraction of this rate he selects at each instant. 
Then we include in the KE 


m = cy, 0<cy<l 
and replace G = afr (see Section A1) in this case by 


oe 


so that a diminished rate of fire is reflected in a current proportionately 
diminished weapon effectiveness. 

Thus P must pick a path of length not exceeding v7, with the flight time 
T, prescribed from his starting point to # so as to minimize the prob- 
ability of his being hit en route. During this flight, E fires at him, 
allocating his ammunition, of fixed amount m, over the interlude so as to 
maximize the hit probability. 

The solution will probably not be of great practical importance, for the 
differences in probability for slightly different routes will likely be nugatory 





[A5] THE BOMBER AND BATTERY GAME 369 


(see the discussion of practical assessment in 4 
Chapter 11). But the instructive level is high: 
the diversity of phenomena is typical of other, 
especially military, problems and there is also 
the illustration of two different type constraints 
effective simultaneously. 

When not otherwise specified, the coast Z 
will be taken as a straight line with the battery 
location O on it. For some purposes it will 
be more enlightening to place O forward of Z; 
we will then speak of advanced defense.* In 
the first instance below we take an arbitrary 
coastline. 

The polar coordinates r, 6 will describe Figure A5.1 
P’s location; they and his control variable 
$ are shown in Figure A5.1. The other state variables are T, the flight 
time allowed P, and m, the ammunition allowed E. Thus the KE are 





F=vsing 
jee 
7 COS $ 
m = —cy, 0O<y<1 
T=-—1. 
Besides the obvious strictures, m > 0, r > 0 on & we demand further: 
vT > rsin 6 (A5.1) 
so that P can always reach £ (at least via PB in the figure) and 
0<0<- 
s 2 


on grounds of symmetry. It is obvious that 0 = 0 will be a dispersal 
surface (but no instantaneous mixed strategy is ever required). 

There are two natural possibilities for the terminal surface: 

@,: Pison Z. Thus the partie is over. There may or may not be some 
excess m. As a parametrization we may take 


r=5s,>0 

6=0 

m=s5,2>0 

T=0. (A5.2) 


S The case of “retarded” defense, where O lies inland, promises little further novelty 
or interest. 
































370 APPENDIX [A5] 


Further, on @,, V = 0, and, in accordance with Section 5.7, we take 


Vp = —4,A>0 


as the third parameter. 

Ga: Whenever m = 0, as there can be no further changes in payoff, 
P’s strategy from here on is immaterial (we suppose T big enough to permit 
his reaching Z, that is, (A5.1)). Thus the plane m = 0 is a natural 
terminal surface. Our parametrization: 


r=s,>0 

6 = Sz, 0<<5 

m=0 

T = si (A5.3) 


The next three examples are instructive but extreme cases in that they 
lie on the fringes of the totality of solutions. 


Example A5.1. No constraint on E’s firepower. We assume m so large 
that E can maintain full fire (y = 1) throughout the partie. Then we may 
ignore m as a state variable. We take initial conditions on @, (with the 
third of (A5.2) suppressed) and G = a/r. 
The problem is thus a pristine illustration 
of the integral constraints discussed in 
Section 5.7 with, of course, L = 1. 

If we solve the system with 2 = 0, we 
obtain a path, which, as we know, would 
be optimal for P even were he freed of 
the constraint on T. For general coasts, 
this path, such as PA in Figure A5.2, turns 
out to have an equation of the form r = 
Cı exp (—c,6) the c, being constants. 

For 4 > 0 we obtain paths (PC,, PC;,... 
in the figure) where the constraint is effective, 
that is, the T allowed P is less than that 
needed to traverse PA. Finally comes the 

N path PB when there is just enough T to 

Figure A5.2 permit travel on a horizontal straight line 
((A5.1) holds as an equality). 

In our standard case of a straight shore with O onit, the absolute optimal 

path PA (see Figure A5.1) reduces to a circular arc of constant r. But the 











[a5] THE BOMBER AND BATTERY GAME 371 


effectively constrained paths are given by 


r= /X@) — 5 


P (A5.4) 


rate ects 
A 149 AX uy X(u) + ali) 
where X(r) = (vr)? + (a/A + s}. 

A closed expression for 0 can be worked out from the general indefinite 
integral a3 
f dx = 1 log Y,(Ce + B) +A + Bz + ay X 

VX(VX +0) 2VB?—C(A—o®)  YxCe + B) +A + Br +a X 
where X = Cr? + 2Bx + A and the Y; are the roots of X — a?. 

The following identity is a consequence of the RPE and ME and useful 
for integrating in almost all cases. 


(rV,) = —(cẸV m + Vr) = constant. (A5.5) 


Example A5.2. No constraint on P’s flight 
time. When P can fly a path of any length, 
but E is constrained by both a maximal 
firing rate c and limited amount m of ammu- 
nition, the optimal play is clear at once: P 
will travel directly away from O until the 
ammunition is exhausted; then he proceeds 
to £ at his leisure. It is also clear that E 
commences firing at full rate (y = 1) at once 


oP eee, 
toee 
ory 
%, 
das 





and so perseveres until exhaustion." P 
Exercise A5.1. Using @,, obtain this result 
analytically. 
Of course, the above solution holds with a : 
Figure A5.3 


constraining T provided it is large enough 
to permit P, after his radial flight, to reach Z without ever decreasing 
the distance OP. Precisely, we must have 


m, 6 vm 
T>—+ 2( F m) : (A5.6) 
c v c 

Equality limits P to the path PAB of Figure A5.3,° AB being a circular arc. 

5 If we were to envisagethis problem removed from context, this solution seems ridicu- 
lous from a practical standpoint. Nevertheless it is mathematically correct. The unreality 
stems from the formulation of the problem, not its solution. Such formulations are a 
hazard for the tyro at military analysis and we urge the reader to reflect on this instance. 

€ Of course, A is part of the terminal conditions @,, and so the arc AB will not be part 
of the formal solution. Nevertheless it will make such an appearance a few pages hence. 
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Example A5.3. With limited ammunition and no maneuvering allowed the 
invader. If (A5.1) holds as an equality, P has no alternative to the straight 
route perpendicular to &. The situation is then identical to that of 
Example 7.14.1 with its semiuniversal surface. 

It is interesting to genéralize to the advanced defense case in which O 
lies a distance D ahead of Z. First suppose m small. Then E should fire 
full blast (y = 1) while x ranges over an interval centered at O of appropri- 
ate length. But if m > 2Dc/v, E has ammunition enough to last while P 
traverses more than the distance 2D. Then E should commence full fire at 
the latest possible time such that he can maintain it until P reaches X. 
That is, he fires during the final interval of play of duration m/c. If < mo|c, 
E opens fire at once; P will reach with some m being surplus. 

All this leads to the paths in &, the (r, m)-plane, the only two state 
variables needed, shown in Figure A5.4, which the reader can readily 
interpret. 


Research Problem A5. Is it possible to derive this manifestly correct 
solution completely by analysis? 


We now investigate more basic solutions and seek a possible y— 
universal surface. The ideas of Chapter 7 require first a recasting into the 
terminal payoff form. The number of state variables is then five; our 
general technique is exceeded, but we can solve this game nonetheless. 

The three equations (7.13.2) become in this case 


CVn +V, =0 (a) 
r 
Vy 
min oY sin ¢ — — cos J — Vr = -p — Vr=0 (8) 
$ r 


Ya sing =0 (y) 
; 


where u is the adjoined state variable. 

Leaving aside the case of V, = 0, obviously trivial, (y) tells us that sin $ 
= 0. Such navigation implies a circular arc path about O with r constant. 

If E had enough ammunition for perpetual full fire, we would be back to 
Example A5.1. Therefore we suppose that y # 1 always. We are faced 
with finding , the optimal way for E to distribute his limited fire power. 

We assert should be constant during the action and this constant rate 
should be such that E consumes all his m just as P reaches Z. For if 
E expended his ammunition nonuniformly, there would have to be at 
least one small interim when yp was greater than the mean rate and another 
in which y was less. If P were to take a short cut during the latter interim, 
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Figure A5.4 


by flying the chord of a small arc of his optimal circular path instead of the 
arc itself, and, further, he uses the J saved thereby to increase r slightly 
during the former interim, then he betters (decreases) the payoff and E’s 
strategy could not be optimal. 

On the alleged universal surface r and » are constants, say sı and sz. 
The parametrization can then be written (in the original &): 


r=58, 
6 = 2 
Sy 
m = CSS 
T=s. (A5.5) 


On this surface, from the original definition of the payoff, 


V= af’ % dt = S, 

or Sy 
Problem 45.1. Show that the necessary conditions («), (8), (y) hold on the 
surface (A5.5). Prove by direct reasoning that it is a universal surface. 


Using (A5.5) as initial conditions, we can integrate the RPE in the 
standard way with y = 0 and 1 to obtain the two sets of tributary paths. 


The tributaries with full fire (p = 1) 

The integration shows that r is always an increasing function of r, 
so that these paths meet the universal surface on its interior side. Note 
also that they are then distinct from those of Example A5.1 where r is 
nonincreasing in 7. 
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Figure A5.5 


Not all the integrals will represent optimal paths. For those with s, = 0, 
which meet. the universal surface at Z with ammunition spent, would 
contradict Example A5.1 were they optimal. 

We examine the limiting behaviour as y > 0. Among the path equations 
with sa = 0 we find 
USs 
s 


r = S — 07, 0 = 


But such are the radial paths of Example A5.2. Further, we see that when 
the universal surface is reached (r = 0), then m = 0 and x continues on the 
circular arc with no firing (y = 0). Thus we have exactly the total path of 
Figure A5.3. This path is unique when E has no surplus T for any other. 
Thus this case marks the blending of this subsection with Example A5.2 as 
T increases through this critical value. 

We now investigate what happens as T begins to fall below the above 
critical amount. The paths progress through the sequence PA,B, of 
Figure A5.5. On PA; there is maximal fire (y = 1); on the circular arcs 
A,B, (i Æ 1), E fires at the constant rate # < 1, which is such as to just 
exhaust the ammunition when P reaches #. 

Let us see what happens at the other limiting extreme as y = s, > 1. 
The tributary paths become circular arcs which lie in the universal surface. 
Such can only happen when 

m= cT. 


Thus there is no break between tributary and universal path. 

To unify our ideas let us see what happens when P starts from a given 
starting point with a fixed but a sufficiently large m and T varies. The 
smallest allowable T admits only the direct path PF of Figure A5.5. 
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Increasing T brings us continuously to the circular arc PD (as in Example 
A5.1) with a decreasing payoff. We suppose there is excess ammunition, 
that is, m > Oat D. Greater T, as we know, leads to the PA,B,. As T is 
decreased in this set we must arrive at the path PA) with A,on Z and Ẹ = 1 
throughout. This path, as we saw, is distinct from PD and is not optimal. 
Thus the payoff for PA, is greater than that for PD. But the payoff for 
PA,(B;) is lower (it is the lowest possible). Therefore there must be an 
intermediate path, say PA,B,, for which V is the same as for PD. 

Thus if T is increased over that for PD, for a certain interval PD remains 
optimal and V stays constant. This happens until T grows enough for P 
to traverse PA,B,; at this state P has two optimal strategies. (Is there a 
dispersal surface?) From then on increasing T leads through the PA;B, 
series with decreasing V until the ultimate at PA. 

The greater m, the greater the gap. With infinite m, as Example A5.1 
shows, PD is the best possible path and the gap has widened into non- 
existence. 


Research Problem A5.2. Characterize the set of starting positions men- 
tioned at which P enjoys two optimal strategies. Is the set attained by a 
construction as in Section 6.5 so that a dispersal surface results? 


The tributaries with no fire (p = 0) 
Integration of the RPE shows that these paths are straight. They then 
must meet the universal surface tangentially and appear as in Figure A5.6. 
The conditions for such are clearly 


r0 > oT 


and m < that required for continuous full fire over the dashed arc of the 
figure. 


v 
ae: 
<= 
pox 


Figure A5.6 
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Figure A5.7 


Observe the plausibility. The path is that which maximizes the minimal 
range occurring on it. 


Finally, there must be still another class of solutions. Consider Example 
A5.3 where vT = r sin 6 so that P is confined to a straight horizontal path 
and m is not great, so that E fires fully in a final interval. We saw a 
resulting semiuniversal surface. 

For a slight increase in T, the path neighbors the straight one and we 
should expect a qualitative similarity. 

Such phenomena are more interesting in the case of advanced defense. 
The chief novelty here is that many optimal parties will end with m 
exhausted before P reaches £ (the reason is intuitively obvious). A @, is 
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suggested which lies left of O and with T just enough to permit a horizontal 
flight to O. 

Two classes of solutions result. One has paths on which r increases with 
t and which curve toward Z. The second entails a transition surface. 
A typical optimal partie appears in Figure A5.7. Here B is a point of the 
above @, and A of the transition surface. The segments PA and BC are 
straight with no fire. The latter occurs only on AB, a curve which meets 
PA and BC smoothly, has (aside from A and B) r < |OA| = |OBt{ and is 
symmetrical about its midpoint. 

Although such paths resemble those of Example A5.3, we have not 
studied the details of the fusion with that case. They probably also merge 
with the # = 0 tributaries of the universal surface. 

The latter surfaces, when the defense is advanced, entail optimal play 
as in Figure A5.8. The arc P,B lies on the universal surface, but it termi- 
nates at a set of points such as B. Left of here the optimal paths are, as 
above, horizontal with no fire. 

The possible blending of the Figure A5.6 tributaries on such universal 
surfaces with the paths of Figure A5.1 is manifestly obvious. 
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Dresher, Melvin, 142, 225 

Duel games, 317 


Economic programming, 87 
E-direction, 70 
Envelope, 113 
Envelope principle, 230, 248 
Escape, 201 
zone, 203, 215 
Escape condition, homicidal chauffeur 
game, 237 
isotropic rocket game, 247, 248 
patrolling a channel, 269 
ES condition, 284 
Euler equations, 95; 173, 189, 351 
Evader, 8, 26 
Existence theorem, 20, 22, 72 
Extensive form, 45 


Feinting, 341 

Firing, game of delayed, and evasion, 
343 

Fixed battery pursuit game, 358 

Flat laxity principle, 350 

Football, 8, 16, 145, 231 

Friction, 105, 244, 248 


Games, discrete differential, 44, 45 
firing, 15 
fusion (degree and kind), 230 
of degree, 8, 12, 13, 35 
of kind, 8, 12, 13, 35, 55, 200, 231 
one-player, 2 
quasi-discrete, 62 
theory of, 1, 10, 45 
zero-sum, 306 
Game with an equivocal surface, 286, 
290 
Geometric method, 144, 260, 273 
Goldschmidt discontinuous solution, 
158 
Grenander, Ulf, 340, 343 
Guarding a target, 10, 19, 29 
Guidance, automatic, 14 
Guided missiles, 361 
Gunner and approaching target, 196 


Hamilton-Jacobi equations, 82 
Hamstrung squad car game, 45, 57 
Headline, 75, 162 
Homicidal chauffeur game, 11, 20, 22, 
23, 70, 84, 135, 217 
asymmetric version, 195 
discrete version, 45, 59, 62 
equivocal phenomenon, 297 
game of kind, 232 
geometric solution, 273 
KE (realistic), 27 
KE (reduced), 29 
primary solution, 278 
US, 193 
Hyperbola, 144 


ICBM trajectory, 362 
Information, 336, 341 
granted, 337 
inferred, 337 
in pursuit games, 338 
Initial conditions, 14, 36, 82, 84 
for SPS, 207 
Instantaneous mixed strategy, 136, 281, 
300 
Interception of straight flying evader, 
223, 255 
followed by early warning, 344 
Involute, 234, 242, 276 
Isoperimetric problem, 129, 131 
Isotropic rocket game, 23, 87, 105, 217, 
244, 250 


Karlin, Samuel, 38 

Kinematic equations, 5, 6, 25, 27, 28, 
312 

K-strategies, 22, 38, 65, 72, 73, 75, 136 


Lagrange multiplier, 131 

Large, in the, 66 

LaSalle, Joseph, 272, 366 

Launching boost, 308 

Lifeline game, 231, 255, 257 

Linear vectogram universal surface, 
analytic condition, 166, 167 

Looping the blockader, 265 


Main equation, 48, 67, 70, 205 
first form (ME,), 68 
second form (ME,), 69 





Manifold, universal, 164 
Matrix, game, 2, 136, 306, 352 
Mengel, Arnold, 96, 330 
Minimax, 36, 68 

Minimax assumption, 35 
Monetropic rocket, 250 
Morgenstern, Oskar, 1, 38 
Moving craft, 5 

Munition(s), 5 


Neumann, John von, 1, 38, 232, 270 
Neutral outcome, 143, 200, 203 
Nonexistence, 65 

(examples), 151, 176 
Nonuseable part, 83 
Normal, 205, 214 


Obstacle tag, 23, 134, 136, 152 
One chance pursuit game, 142, 225 


Parking a car, 174 
Path equations, 80 
Path, optimal, 21, 48, 64, 70 
tributary, 134, 177, 196, 280 
Patrol line, 231, 269 
circle, 269 
Patrolling a channel (corridor game), 
266 
Payoff, 4, 8, 14, 32, 68, 203, 311, 318 
firepower, 318, 319 
hit probability, 357 
integral, 32 
manpower, 318, 321 
terminal, 32, 43, 47 
P-direction, 70 
Perpetuated dilemma, 137, 149 
Players, 8, 26, 27 
Policy, 39 
method of explicit, 200, 202 
Primary, 278, 282 
Princess and monster game, 349 
simpler, 350 
Probabilistic indefiniteness, 352 
Probability, hit, 357 
kill, 196 
Problems, unsolved, 23 
Programming, 15 
Pursuer, 8, 26, 27 
Pursuit games, 8, 10, 26, 27, 39, 45, 56, 
200, 218 
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Pursuit games, bounded, 270 
Pursuit, pure, 297 


Resource, 125 

Retrograde path equations, 82, 206 
Retrogression principle, 81 
Retrogressive form, 48 


Safe region, 144 
boundary of, 144 
Sandpiles, 154 
Satellite orbit, 366 
Search games, 336 
with immobile hiders, 345 
with mobile hiders, 349 
Secondary, 282 
Semipermeability, 71, 204 
Shortest route of car, boat, airplane, 
170 
Simple aiming and evading game, 355 
Simple blocking game, 147, 260 
Simple motion, 7, 27, 39 
Simple search game, 345, 348 
Simplest pursuit game, 16, 144, 342 
Simulation (war games), 307 
Small, in the, 66, 84 
Solution, 64 
Space, playing or state, 3, 25 
realistic, 28 
reduced, 28, 29, 30, 57, 59 
Stability, 272 
State, 45 
State variables, 3, 6, 26 
Static (SPS), 205 
Stationary (steady state), 336, 343 
Steel production, optimal program of, 
116 
Stochastic processes, 340, 343 
Stop rule, 31, 35 
Strategy(ies), 14, 36, 39, 48 
all optimal, 175 
instantaneous mixed, 136 
K-, 22, 38, 65, 72, 73, 75, 136 
mixed, 78, 336, 337 
optimal, 64, 69, 202 
penetrating, 283 
traversing, 282 
Surface, 21, 25, 32 
afferent, 134, 156 
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Surface, dispersal, 132, 134, 152 
construction of, 140 
efferent, 132 
equivocal, 273, 282, 284, 295 
condition (ES), 284 
semipermeable, 70, 204 
construction of, 205 
semiuniversal, 196, 367 
singular, 21, 66, 67, 132, 209 
classification of, 132 
termination, 4, 30, 48 
transition, 87, 187 
test for, 188 
universal, 24, 134, 156, 173, 189 
control variables restored, 190 
linear vectogram, 161, 166 
necessary condition, 187 
null integrand, 157 
Surface of revolution of least area, 158 
two handed version, 161 
Swerve, 13, 22, 87, 105, 116, 249, 297 


Tactic, 38, 73, 75 
Tenet of transition, 67 
Termination, 4, 30, 31, 43, 75 
Thrust, 105 

optimal direction of, 365 


Tic-tac-toe, 307 
Trajectories, ICBM, 362 

optimal, for guided missiles, 361 
Transformations on variables, 40 
Tributary (ies), 134, 177, 280 
Two cars, game of, 21, 202, 203, 237 


Useable part, 83 


Value, 4, 16, 36, 47, 48, 64, 67 
Variations, calculus of, 16, 23, 66, 86, 
96, 124, 157, 173, 189, 273, 
282, 351 
Vectogram, 26, 37, 42 
canonization, 40 
circular, 43 
linear, 150 
universal surfaces, 161 
Verification theorem, 22, 72 
Void, 169, 187 


Wall pursuit game, 137, 149, 255 
War, warfare, 305, 329 
Workable condition for linear vecto- 
gram US, n = 3, 168, 191 
n = 4,177 
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